Deployed b97b20b with MkDocs version: 1.3.0
This commit is contained in:
parent
38c201a13f
commit
9182ba84ed
35 changed files with 11722 additions and 1478 deletions
|
|
@ -1,16 +1,19 @@
|
|||
# Train and deploy a SOTA model
|
||||
|
||||
Let's see GreatAI in action by going over the life-cycle of a simple service.
|
||||
|
||||
## Objectives
|
||||
|
||||
1. You will see how the [great_ai.utilities][] can integrate into your Data Science workflow.
|
||||
2. You will use [great_ai.large_file][] to version and store your trained model.
|
||||
3. You will use [GreatAI][great_ai.GreatAI] to prepare your model for a robust and responsible deployment.
|
||||
|
||||
## Overview
|
||||
|
||||
You are going to train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the [SciBERT paper](https://arxiv.org/abs/1903.10676) in which SciBERT [achieved an F1-score of 0.6571](https://paperswithcode.com/sota/sentence-classification-on-paper-field). We are going to outperform it using a trivial text classification model: a [Linear SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html).
|
||||
|
||||
We use the same synthetic dataset derived from the [Microsoft Academic Graph](https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/). The dataset is [available here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag).
|
||||
|
||||
## Objectives
|
||||
|
||||
1. You will see how the `great_ai.utilities` can integrate into your Data Science workflow.
|
||||
2. You will use `great_ai.large_file` to version and store your trained model
|
||||
3. You will use `GreatAI` to prepare your model for a robust and responsible deployment
|
||||
|
||||
!!! success
|
||||
You are ready to start the tutorial. Feel free to come back to the [summary](#summary) section once you're finished.
|
||||
|
|
@ -18,7 +21,7 @@ We use the same synthetic dataset derived from the [Microsoft Academic Graph](ht
|
|||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||
[:fontawesome-solid-chart-simple: Train it](train.ipynb){ .md-button .md-button--primary }
|
||||
|
||||
[:material-cloud-tags: Deploy it](deploy.ipynb){ .md-button .md-button--secondary }
|
||||
[:material-cloud-tags: Deploy it](deploy.ipynb){ .md-button .md-button--primary }
|
||||
</div>
|
||||
|
||||
|
||||
|
|
@ -26,11 +29,11 @@ We use the same synthetic dataset derived from the [Microsoft Academic Graph](ht
|
|||
|
||||
### The [training notebook](train.ipynb)
|
||||
|
||||
We load and preprocess the dataset while relying on `great_ai.utilities.clean` for the heavy-lifting. Additionally, the preprocessing is parallelised using `great_ai.utilities.simple_parallel_map`.
|
||||
We load and preprocess the dataset while relying on [great_ai.utilities.clean][] for the heavy-lifting. Additionally, the preprocessing is parallelised using [great_ai.utilities.simple_parallel_map][]
|
||||
|
||||
After training and evaluating a model, it is exported using `great_ai.save_model`.
|
||||
After training and evaluating a model, it is exported using [great_ai.save_model][].
|
||||
|
||||
??? tip "Using the cloud"
|
||||
??? tip "Remote storage"
|
||||
To store your model remotely, you need to set your credentials before calling `save_model`.
|
||||
|
||||
For example, to use [AWS S3](https://aws.amazon.com/s3/):
|
||||
|
|
@ -51,7 +54,7 @@ After training and evaluating a model, it is exported using `great_ai.save_model
|
|||
|
||||
### The [deployment notebook](deploy.ipynb)
|
||||
|
||||
We create an inference function that can be hardened by wrapping it in a `great_ai.GreatAI` instance.
|
||||
We create an inference function that can be hardened by wrapping it in a [GreatAI][great_ai.GreatAI] instance.
|
||||
|
||||
```python
|
||||
from great_ai import GreatAI, use_model
|
||||
|
|
@ -64,7 +67,7 @@ def predict_domain(sentence, model):
|
|||
return str(model.predict(inputs)[0])
|
||||
```
|
||||
|
||||
1. `@use_model` loads and injects your model into the `predict_domain` function's `model` argument.
|
||||
1. [@use_model][great_ai.use_model] loads and injects your model into the `predict_domain` function's `model` argument.
|
||||
You can freely reference it knowing that it is always given to the function.
|
||||
|
||||
Finally, we deploy the model, inference function, and the GreatAI wrapping all of these. For that we either use: `great-ai deploy.ipynb` or build a Docker image.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue