Fix typos
This commit is contained in:
parent
aa548f0074
commit
aa31d2bb0a
7 changed files with 13 additions and 15 deletions
|
|
@ -103,9 +103,7 @@
|
||||||
" unchunk,\n",
|
" unchunk,\n",
|
||||||
")\n",
|
")\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
|
||||||
"def preprocess_chunk(chunk_key: str) -> List[Tuple[str, List[str]]]:\n",
|
"def preprocess_chunk(chunk_key: str) -> List[Tuple[str, List[str]]]:\n",
|
||||||
" # Extract\n",
|
|
||||||
" response = urllib.request.urlopen(\n",
|
" response = urllib.request.urlopen(\n",
|
||||||
" f\"https://s3-us-west-2.amazonaws.com/ai2-s2-research-public/\"\n",
|
" f\"https://s3-us-west-2.amazonaws.com/ai2-s2-research-public/\"\n",
|
||||||
" \"open-corpus/2022-02-01/{chunk_key}\"\n",
|
" \"open-corpus/2022-02-01/{chunk_key}\"\n",
|
||||||
|
|
|
||||||
|
|
@ -11,7 +11,7 @@
|
||||||
"\n",
|
"\n",
|
||||||
"In the first part, we have cleaned and transformed our training data. We can now access this data using `great_ai.LargeFile`. Locally, it will gives us the cached version, otherwise, the latest version is downloaded from S3 or GridFS. \n",
|
"In the first part, we have cleaned and transformed our training data. We can now access this data using `great_ai.LargeFile`. Locally, it will gives us the cached version, otherwise, the latest version is downloaded from S3 or GridFS. \n",
|
||||||
"\n",
|
"\n",
|
||||||
"In this part, we hyperparameter-optimise and train a simple, Naive Bayes classifier which we then export for deployment using `great_ai.save_model`.\n",
|
"In this part, we hyperparameter-optimise and train a simple Naive Bayes classifier which we then export for deployment using `great_ai.save_model`.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"## Load data that has been extracted in [part 1](/examples/simple/data)"
|
"## Load data that has been extracted in [part 1](/examples/simple/data)"
|
||||||
]
|
]
|
||||||
|
|
|
||||||
|
|
@ -35,7 +35,7 @@ As you can see, the original return value is wrapped in a [Trace][great_ai.Trace
|
||||||
|
|
||||||
Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
|
Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
|
||||||
|
|
||||||
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To this end, `uvicorn` is wrapped by the `great-ai` command-line utility which, among others, takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
|
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, `uvicorn` is wrapped by the `great-ai` command-line utility which, among others, takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
|
||||||
|
|
||||||
### In development
|
### In development
|
||||||
|
|
||||||
|
|
@ -81,7 +81,7 @@ docker run -p 6060:6060 --volume `pwd`:/app --rm \
|
||||||
|
|
||||||
#### Use a Platform-as-a-Service
|
#### Use a Platform-as-a-Service
|
||||||
|
|
||||||
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [AWS ECS](https://aws.amazon.com/ecs/){ target=_blank }, [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank }, [MLEM](https://mlem.ai/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
|
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [AWS ECS](https://aws.amazon.com/ecs/){ target=_blank }, [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank }, [MLEM](https://mlem.ai/){ target=_blank }, [Streamlit](https://streamlit.io/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
|
||||||
|
|
||||||
To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.
|
To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.
|
||||||
|
|
||||||
|
|
@ -100,7 +100,7 @@ RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor
|
||||||
# Add you application code to the image
|
# Add you application code to the image
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
# The default ENTRYPOINT is great-ai, specify it's argument using CMD
|
# The default ENTRYPOINT is great-ai, specify its argument using CMD
|
||||||
CMD ["deploy.ipynb"]
|
CMD ["deploy.ipynb"]
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
@ -86,7 +86,7 @@ great-ai demo.py
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
GreatAI fits between the prototype and deployment phases of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented aiming to achieve the following attributes:
|
GreatAI fits between the prototype and deployment phases of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, several best practices can be automatically implemented aiming to achieve the following attributes:
|
||||||
|
|
||||||
- **G**eneral: use any Python library without restriction
|
- **G**eneral: use any Python library without restriction
|
||||||
- **R**obust: have error-handling and well-tested utilities out-of-the-box
|
- **R**obust: have error-handling and well-tested utilities out-of-the-box
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@
|
||||||
"source": [
|
"source": [
|
||||||
"# Harden and deploy your app\n",
|
"# Harden and deploy your app\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Finally, it's time to deploy your model. But before that, you have to make sure you follow AI deployment [best-practices](https://se-ml.github.io/). In the past, this step was too often either the source of unexpected struggles, or worse, simply ignored.\n",
|
"Finally, it's time to deploy your model. But before that, you have to make sure you follow AI deployment [best-practices](https://se-ml.github.io/). In the past, this step was too often either the source of unexpected struggle, or worse, simply ignored.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"With `GreatAI`, it has become a matter of 4 lines of code."
|
"With `GreatAI`, it has become a matter of 4 lines of code."
|
||||||
]
|
]
|
||||||
|
|
|
||||||
|
|
@ -5,14 +5,14 @@ Let's see `great-ai` in action by going over the life-cycle of a simple service.
|
||||||
## Objectives
|
## Objectives
|
||||||
|
|
||||||
1. You will see how the [great_ai.utilities](/reference/utilities) can integrate into your Data Science workflow.
|
1. You will see how the [great_ai.utilities](/reference/utilities) can integrate into your Data Science workflow.
|
||||||
2. You will use [great_ai.large_file](/reference/large_file) to version and store your trained model.
|
2. You will use [great_ai.large_file](/reference/large-file) to version and store your trained model.
|
||||||
3. You will use [GreatAI][great_ai.GreatAI] to prepare your model for a robust and responsible deployment.
|
3. You will use [GreatAI][great_ai.GreatAI] to prepare your model for a robust and responsible deployment.
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
You are going to train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the [SciBERT paper](https://arxiv.org/abs/1903.10676) in which SciBERT [achieved an F1-score of 0.6571](https://paperswithcode.com/sota/sentence-classification-on-paper-field). We are going to outperform it using a trivial text classification model: a [Linear SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html).
|
You are going to train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the [SciBERT paper](https://arxiv.org/abs/1903.10676){ target=_blank } in which SciBERT [achieved an F1-score of 0.6571](https://paperswithcode.com/sota/sentence-classification-on-paper-field){ target=_blank }. We are going to outperform it using a trivial text classification model: a [Linear SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html){ target=_blank }.
|
||||||
|
|
||||||
We use the same synthetic dataset derived from the [Microsoft Academic Graph](https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/). The dataset is [available here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag).
|
We use the same synthetic dataset derived from the [Microsoft Academic Graph](https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/){ target=_blank }. The dataset is [available here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag){ target=_blank }.
|
||||||
|
|
||||||
!!! success
|
!!! success
|
||||||
You are ready to start the tutorial. Feel free to come back to the [summary](#summary) section once you're finished.
|
You are ready to start the tutorial. Feel free to come back to the [summary](#summary) section once you're finished.
|
||||||
|
|
@ -70,7 +70,7 @@ def predict_domain(sentence, model):
|
||||||
1. [@use_model][great_ai.use_model] loads and injects your model into the `predict_domain` function's `model` argument.
|
1. [@use_model][great_ai.use_model] loads and injects your model into the `predict_domain` function's `model` argument.
|
||||||
You can freely reference it knowing that the function is always provided with it.
|
You can freely reference it knowing that the function is always provided with it.
|
||||||
|
|
||||||
Finally, we test the model's inference function through the GreatAI dashboard. [The only thing left is to deploy the hardened-service properly.](/how-to-guides/use-service)
|
Finally, we test the model's inference function through the GreatAI dashboard. [The only thing left is to deploy the hardened service properly.](/how-to-guides/use-service)
|
||||||
|
|
||||||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||||
[:material-book: Learn about all the features](/how-to-guides/create-service){ .md-button .md-button--primary }
|
[:material-book: Learn about all the features](/how-to-guides/create-service){ .md-button .md-button--primary }
|
||||||
|
|
|
||||||
|
|
@ -30,9 +30,9 @@
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"First, we have to get some data. After downloading it [from here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag), we might notice that the dataset is in [JSON Lines](https://jsonlines.org/) format (each line is a seperate JSON document). \n",
|
"First, we have to get some data. After downloading it [from here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag), we might notice that the dataset is in [JSON Lines](https://jsonlines.org/) format (each line is a separate JSON document). \n",
|
||||||
"\n",
|
"\n",
|
||||||
"Let's write a function which takes a single line, and returns the sentence and the corresponding label from it. Before returning, the sentence is also [cleaned](/reference/utilities/#great_ai.utilities.clean.clean) to remove any LaTeX, XML, unicode, PDF-extraction artifacts."
|
"Let's write a function which takes a single line and returns the sentence and the corresponding label from it. Before returning, the sentence is also [cleaned](/reference/utilities/#great_ai.utilities.clean.clean) to remove any LaTeX, XML, unicode, PDF-extraction artifacts."
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|
@ -238,7 +238,7 @@
|
||||||
"\n",
|
"\n",
|
||||||
"You might wonder that *\"this is great, but besides some utility functions (`clean`, `simple_parallel_map`, ...) what more value does GreatAI add?\"*. This would be a valid argument because the scope of GreatAI actually only starts here.\n",
|
"You might wonder that *\"this is great, but besides some utility functions (`clean`, `simple_parallel_map`, ...) what more value does GreatAI add?\"*. This would be a valid argument because the scope of GreatAI actually only starts here.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"> Not coincidentally, this is the point where the scope of Data Science ends but it's still a grey-zone for software engineering.\n",
|
"> Not coincidentally, this is the point where the scope of Data Science ends but it's still a grey zone for software engineering.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"In order to use this model in production, we have to make it available on some possibly shared infrastructure."
|
"In order to use this model in production, we have to make it available on some possibly shared infrastructure."
|
||||||
]
|
]
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue