Add more documentation
This commit is contained in:
parent
7165174f4f
commit
8d1ad9a242
21 changed files with 406 additions and 92 deletions
|
|
@ -0,0 +1,2 @@
|
|||
# Call remote GreatAI instances
|
||||
|
||||
101
docs/how-to-guides/configure-service.md
Normal file
101
docs/how-to-guides/configure-service.md
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
# How to configure GreatAI
|
||||
|
||||
GreatAI aims to provide reasonable defaults wherever possible. The current configuration is always prominently displayed (and updated) on the dashboard and in the command-line startup banner.
|
||||
|
||||
## Using [great_ai.configure][]
|
||||
|
||||
You can override any of the default settings by calling [great_ai.configure][]. If you don't call `configure`, the default settings are applied on the first call to most `great-ai` functions.
|
||||
|
||||
!!! warning
|
||||
You must call [great_ai.configure][] before calling (or decorating with) any other `great-ai` function. However, importing other functions before calling [great_ai.configure][] is permitted.
|
||||
|
||||
```python title="configure-demo.py"
|
||||
from great_ai import configure, RouteConfig
|
||||
import logging
|
||||
|
||||
configure(
|
||||
version='1.0.0',
|
||||
log_level=logging.INFO,
|
||||
seed=2,
|
||||
should_log_exception_stack=False,
|
||||
prediction_cache_size=0, #(1)
|
||||
disable_se4ml_banner=True,
|
||||
dashboard_table_size=200,
|
||||
route_config=RouteConfig( #(2)
|
||||
feedback_endpoints_enabled=False,
|
||||
dashboard_enabled=False
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
1. Completely disable caching.
|
||||
2. The unspecified routes are enabled by default.
|
||||
|
||||
## Using remote storage
|
||||
|
||||
The only aspect that cannot be automated is choosing the backing storage for the database and file storage.
|
||||
|
||||
Right now, you have 3 options for storing the models and large datasets: [great_ai.large_file.LargeFileLocal][], [great_ai.large_file.LargeFileMongo][], and [great_ai.large_file.LargeFileS3][].
|
||||
|
||||
Without explicit configuration, [great_ai.large_file.LargeFileLocal][] is selected by default. This one still version-controls your files but it only stores them in a local path.
|
||||
|
||||
!!! important
|
||||
If your working directory contains a `mongo.ini` or `s3.ini` file, an attempt is made to auto-configure [LargeFileMongo][great_ai.large_file.LargeFileMongo] or [LargeFileS3][great_ai.large_file.LargeFileS3] respectively.
|
||||
|
||||
To use [LargeFileMongo][great_ai.large_file.LargeFileMongo] or [LargeFileS3][great_ai.large_file.LargeFileS3] explicitly, configure them before calling any other `great-ai` function.
|
||||
|
||||
### S3-compatible
|
||||
|
||||
```toml title="s3.ini"
|
||||
aws_region_name = eu-west-2
|
||||
aws_access_key_id = MY_AWS_ACCESS_KEY # ENV:MY_AWS_ACCESS_KEY would also work
|
||||
aws_secret_access_key = MY_AWS_SECRET_KEY
|
||||
large_files_bucket_name = bucket-for-models
|
||||
```
|
||||
|
||||
```python title="use-s3.py"
|
||||
from great_ai.large_file import LargeFileS3
|
||||
from great_ai import save_model
|
||||
|
||||
LargeFileS3.configure_credentials_from_file('s3.ini') #(1)
|
||||
|
||||
model = [4, 3]
|
||||
save_model(model, 'my-model')
|
||||
```
|
||||
|
||||
1. This line isn't strictly necceseary because if `s3.ini` (or `mongo.ini`) is available in the current working directory, they are automatically used to configure their respective LargeFile implementations/databases.
|
||||
|
||||
??? note "Departing from AWS"
|
||||
With the `aws_endpoint_url` argument, it is possible to use any other S3-compatible service such as [Backblaze](https://www.backblaze.com/){ target=_blank }. In that case, it would be `aws_endpoint_url=https://s3.us-west-002.backblazeb2.com`.
|
||||
|
||||
### GridFS
|
||||
|
||||
[GridFS](https://www.mongodb.com/docs/manual/core/gridfs/#:~:text=GridFS%20is%20a%20specification%20for,chunk%20as%20a%20separate%20document.){ target=_blank } specifies how to store files in MongoDB. The official MongoDB server and many compatible implementations support it.
|
||||
|
||||
```toml title="mongo.ini"
|
||||
MONGO_CONNECTION_STRING=mongodb://localhost:27017 # this is the default value
|
||||
# if `MONGO_CONNECTION_STRING` is specified, this default is overridden
|
||||
MONGO_CONNECTION_STRING=ENV:MONGO_CONNECTION_STRING
|
||||
MONGO_DATABASE=my-database # it is automatically created if doesn't exist
|
||||
```
|
||||
|
||||
```python title="use-mongo.py"
|
||||
from great_ai.large_file import LargeFileMongo
|
||||
from great_ai import save_model
|
||||
|
||||
LargeFileMongo.configure_credentials_from_file('mongo.ini')
|
||||
|
||||
model = [4, 3]
|
||||
save_model(model, 'my-model')
|
||||
```
|
||||
|
||||
!!! note "Simplifying config files"
|
||||
You can combine `mongo.ini` or `s3.ini` with your application's config file because the unneeded keys are ignored by the `configure_credentials_from_file` method.
|
||||
|
||||
## Using a database
|
||||
|
||||
By default, a thread-safe version of [TinyDB](https://tinydb.readthedocs.io/en/latest/){ target=_blank } is utilised for saving the prediction traces into a local file. Unfortunately, for most production needs, this method is not suitable.
|
||||
|
||||
### MongoDB
|
||||
|
||||
At the moment, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory, or explicitly call [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver.configure_credentials_from_file] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# How to create a GreatAI service
|
||||
|
||||
The core value of `great-ai` lies in its [GreatAI][great_ai.deploy.GreatAI] class. In order to take advantage of it, you need to create an instance wrapping your code.
|
||||
The core value of `great-ai` lies in its [GreatAI][great_ai.GreatAI] class. In order to take advantage of it, you need to create an instance wrapping your code.
|
||||
|
||||
Let's say that you have the following greeter function:
|
||||
|
||||
|
|
@ -9,7 +9,7 @@ def my_greeter_function(your_name):
|
|||
return f'Hi {your_name}!'
|
||||
```
|
||||
|
||||
You can simply decorate (wrap) this function using the [@GreatAI.create][great_ai.deploy.GreatAI.create] factory.
|
||||
You can simply decorate (wrap) this function using the [@GreatAI.create][great_ai.GreatAI.create] factory.
|
||||
|
||||
```python title="greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
|
@ -20,7 +20,7 @@ def greeter(your_name):
|
|||
```
|
||||
|
||||
??? info "Why not simply use `@GreatAI?`"
|
||||
The purpose of the [@GreatAI.create][great_ai.deploy.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
|
||||
The purpose of the [@GreatAI.create][great_ai.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
|
||||
|
||||
## With types
|
||||
|
||||
|
|
@ -52,9 +52,9 @@ async def async_greeter(your_name: str) -> str:
|
|||
|
||||
## With decorators
|
||||
|
||||
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.deploy.GreatAI.create] always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
|
||||
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.GreatAI.create] always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
|
||||
|
||||
### Using `use_model`
|
||||
### Using `@use_model`
|
||||
|
||||
If you have previously saved a model with [save_model][great_ai.save_model], you can inject it into your function by calling [@use_model][great_ai.use_model].
|
||||
|
||||
|
|
@ -72,14 +72,14 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
|
|||
1. By default, the parameter named `model` will be replaced by the loaded model. This behaviour can be customised by setting the `model_kwarg_name`. This way, even multiple models can be injected into a single function.
|
||||
|
||||
!!! important
|
||||
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.deploy.GreatAI.create]. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.GreatAI.create]. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
|
||||
### Using `parameter`
|
||||
### Using `@parameter`
|
||||
|
||||
If you wish to turn of logging or specify custom validation for your parameters, you can use the [@parameter][great_ai.parameter] decorator.
|
||||
|
||||
!!! note
|
||||
By default, all parameters that are not affected by an explicit [@parameter][great_ai.parameter] or [@use_model][great_ai.use_model] are automatically decorated with [@parameter][great_ai.parameter] when [@GreatAI.create][great_ai.deploy.GreatAI.create] is called.
|
||||
By default, all parameters that are not affected by an explicit [@parameter][great_ai.parameter] or [@use_model][great_ai.use_model] are automatically decorated with [@parameter][great_ai.parameter] when [@GreatAI.create][great_ai.GreatAI.create] is called.
|
||||
|
||||
```python title="greeter_with_validation.py"
|
||||
from great_ai import GreatAI, use_model
|
||||
|
|
@ -93,7 +93,7 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
|
|||
```
|
||||
|
||||
!!! important
|
||||
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.deploy.GreatAI.create]. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.GreatAI.create]. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
|
||||
## Complex example
|
||||
|
||||
|
|
|
|||
6
docs/how-to-guides/handle-training-data.md
Normal file
6
docs/how-to-guides/handle-training-data.md
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
# How to handle training data
|
||||
|
||||
## Upload data
|
||||
|
||||
## Use feedback
|
||||
|
||||
118
docs/how-to-guides/large_file.md
Normal file
118
docs/how-to-guides/large_file.md
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
# How to use LargeFile
|
||||
|
||||
Storing, versioning, and downloading files from S3 made as easy as using `open()` in Python. Caching included.
|
||||
|
||||
## Motivation
|
||||
|
||||
Oftentimes, especially when working with data-heavy applications, large files can proliferate in a repository. Version controlling them is an obvious next step, however, GitHub's git LFS implementation [doesn't support the deletion](https://docs.github.com/en/repositories/working-with-files/managing-large-files/removing-files-from-git-large-file-storage#git-lfs-objects-in-your-repository) of large files, making it easy for them to eat-up the LFS quota and explode the size of your repos.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
pip install open-large
|
||||
```
|
||||
|
||||
### Simple example
|
||||
|
||||
```python
|
||||
from great_ai.large_file import LargeFileS3
|
||||
|
||||
LargeFileS3.configure_credentials({
|
||||
"aws_region_name": "your_region_like_eu-west-2",
|
||||
"aws_access_key_id": "YOUR_ACCESS_KEY_ID",
|
||||
"aws_secret_access_key": "YOUR_VERY_SECRET_ACCESS_KEY",
|
||||
"large_files_bucket_name": "create_a_bucket_and_put_its_name_here",
|
||||
})
|
||||
|
||||
# Creates a new version and deletes the older version leaving the 3 most recently used intact
|
||||
with LargeFileS3("test.txt", "w", keep_last_n=3) as f:
|
||||
for i in range(100000):
|
||||
f.write('test\n')
|
||||
|
||||
# By default the latest version is returned
|
||||
# but an optional `version` keyword argument can be provided as well
|
||||
with LargeFileS3("test.txt", "r") as f:
|
||||
print(f.readlines()[0])
|
||||
```
|
||||
|
||||
> Automatically creates a file, writes to it, uploads it to S3, and then queries the most recent version of it.
|
||||
> In this case, the latest version is already in the local cache, no download is required.
|
||||
|
||||
### More details
|
||||
|
||||
`LargeFile` behaves like an opened file (in the background it is a temp file after all). Binary reading and writing is supported along with the [different keywords](https://docs.python.org/3/library/functions.html#open) `open()` accepts.
|
||||
|
||||
The local cache can be configured with these properties:
|
||||
|
||||
```python
|
||||
LargeFile.cache_path = Path('.cache')
|
||||
LargeFile.max_cache_size = "30 GB"
|
||||
```
|
||||
|
||||
#### I only need a path
|
||||
|
||||
In case you only need a path to the "remote" file, this pattern can be applied:
|
||||
|
||||
```python
|
||||
path_to_model = LargeFile("folder-of-my-bert-model", version=31).get()
|
||||
```
|
||||
|
||||
> This will first download the file/folder into your local cache folder. Then, it returns a `Path` object to the local version. Which can be turned into a string with `str(path_to_model)`.
|
||||
|
||||
The same approach works for uploads:
|
||||
|
||||
```python
|
||||
LargeFile("folder-of-my-bert-model").push('path_to_local/folder_or_file')
|
||||
```
|
||||
|
||||
> This way, both regular files and folders can be handled. The uploaded file is called **folder-of-my-bert-model**, the local name is ignored.
|
||||
|
||||
Lastly, all version of the remote object can be deleted by calling `LargeFile("my-file").delete()`. It will still reside in your local cache afterwards, its deletion will happen next time your local cache has to be pruned.
|
||||
|
||||
### Command-line example
|
||||
|
||||
The package can be used as a module from the command-line to give you more flexibility.
|
||||
|
||||
#### Setup
|
||||
|
||||
Create an .ini file (or use _~/.aws/credentials_). It may look like this:
|
||||
|
||||
```ini
|
||||
aws_region_name = your_region_like_eu-west-2
|
||||
aws_access_key_id = YOUR_ACCESS_KEY_ID
|
||||
aws_secret_access_key = YOUR_VERY_SECRET_ACCESS_KEY
|
||||
large_files_bucket_name = my_large_files
|
||||
endpoint_url = this is optional, for backblaze, use this: https://s3.us-west-002.backblazeb2.com
|
||||
```
|
||||
|
||||
> Just like in [example secrets](example_secrets.ini).
|
||||
|
||||
#### Print the expected options
|
||||
|
||||
```sh
|
||||
python3 -m large_file --help
|
||||
```
|
||||
|
||||
#### Upload some files
|
||||
|
||||
```sh
|
||||
python3 -m large_file --backend s3 --secrets secrets.ini --push my_first_file.json folder/my_second_file my_folder
|
||||
```
|
||||
|
||||
> Only the filename is used as the S3 name, the rest of the path is ignored.
|
||||
|
||||
#### Download some files to the local cache
|
||||
|
||||
This can be useful when building a Docker image for example. This way, the files can already reside inside the container and need not be downloaded later.
|
||||
|
||||
```sh
|
||||
python3 -m large_file --backend s3 -secrets ~/.aws/credentials --cache my_first_file.json:3 my_second_file my_folder:0
|
||||
```
|
||||
|
||||
> Versions may be specified by using `:`-s.
|
||||
|
||||
#### Delete remote files
|
||||
|
||||
```sh
|
||||
python3 -m large_file --backend s3 --secrets ~/.aws/credentials --delete my_first_file.json
|
||||
```
|
||||
|
|
@ -16,13 +16,88 @@
|
|||
") as client:\n",
|
||||
" pass"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | GreatAI (v0.1.3): configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 tracing_database: ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 large_file_implementation: LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 dashboard_table_size: 50\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | > Find out more at https://se-ml.github.io/practices\u001b[0m\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Trace[str]({'created': '2022-07-11T17:00:27.064568',\n",
|
||||
" 'exception': None,\n",
|
||||
" 'feedback': None,\n",
|
||||
" 'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},\n",
|
||||
" 'models': [],\n",
|
||||
" 'original_execution_time_ms': 0.0896,\n",
|
||||
" 'output': 'Hi Bob',\n",
|
||||
" 'tags': ['greeter', 'online', 'development'],\n",
|
||||
" 'trace_id': '8ff5b268-2613-4e85-96ae-f248666a051f'})"
|
||||
]
|
||||
},
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from great_ai import configure, RouteConfig\n",
|
||||
"from great_ai import GreatAI\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@GreatAI.create\n",
|
||||
"def greeter(your_name: str) -> str:\n",
|
||||
" return f\"Hi {your_name}\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"greeter(\"Bob\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.10.4 ('.env': venv)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.4"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "02dd6d3afbfa9fbbe1037d64ad9014965528a1ccad21929d6e72f466389a68ad"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
|
|
|
|||
131
docs/how-to-guides/use-service.md
Normal file
131
docs/how-to-guides/use-service.md
Normal file
|
|
@ -0,0 +1,131 @@
|
|||
# How to use a GreatAI service
|
||||
|
||||
After [creating a GreatAI service](/how-to-guides/create-service) by wrapping your prediction function, and optionally [configuring it](/how-to-guides/configure-service), it's time to do some prediction.
|
||||
|
||||
Let's take the following example:
|
||||
|
||||
```python title="greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
||||
@GreatAI.create
|
||||
def greeter(your_name: str) -> str:
|
||||
return f'Hi {your_name}'
|
||||
```
|
||||
|
||||
## One-off prediction
|
||||
|
||||
Even though `greeter` is now an instance of [GreatAI][great_ai.GreatAI], you can continue using it as a regular function.
|
||||
|
||||
```python
|
||||
>>> greeter('Bob')
|
||||
Trace[str]({'created': '2022-07-11T14:31:46.183764',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.0381,
|
||||
'output': 'Hi Bob',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': '7c284fd7-7f0d-4464-b5f8-3ef126df34af'})
|
||||
```
|
||||
|
||||
As you can see, the original return value is wrapped in a [Trace][great_ai.Trace] object (which is also persisted in your database of choice). You can access the original value under the `output` property.
|
||||
|
||||
## Online prediction
|
||||
|
||||
Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
|
||||
|
||||
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To this end, `uvicorn` is wrapped by the `great-ai` command-line utility which, among others, takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
|
||||
|
||||
### In development
|
||||
|
||||
```sh
|
||||
great-ai greeter.py
|
||||
```
|
||||
|
||||
!!! success
|
||||
Your model is accessible at [localhost:6060](http:/127.0.0.1:6060){ target=_blank }.
|
||||
|
||||
Some configuration options are also supported.
|
||||
|
||||
```sh
|
||||
great-ai greeter.py --port 8000 --host 127.0.0.1 --timeout_keep_alive 10
|
||||
```
|
||||
> For more options (but no Notebook support, use [uvicorn](https://www.uvicorn.org/){ target=_blank })
|
||||
|
||||
### In production
|
||||
|
||||
There are three main approaches for deploying a GreatAI service.
|
||||
|
||||
#### Manual deployment
|
||||
|
||||
The app is run in *production-mode* if the value of the `ENVIRONMENT` environment variable is set to `production`.
|
||||
|
||||
```sh
|
||||
ENVIRONMENT=production great-ai greeter.py
|
||||
```
|
||||
|
||||
Simply run `ENVIRONMENT=production great-ai deploy.ipynb` in the command-line of a production machine.
|
||||
> This is the crudest approach, however, it might be fitting for some contexts.
|
||||
|
||||
#### Containerised deployment
|
||||
|
||||
Run the notebook directly in a container or create a service for it using your favourite container orchestrator.
|
||||
|
||||
```sh
|
||||
docker run -p 6060:6060 --volume `pwd`:/app --rm \
|
||||
schmelczera/great-ai deploy.ipynb
|
||||
```
|
||||
> You can replace ``pwd`` with the path to your code's folder.
|
||||
|
||||
#### Use a Platform-as-a-Service
|
||||
|
||||
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank } or [MLEM](https://mlem.ai/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
|
||||
|
||||
To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.
|
||||
|
||||
```Dockerfile
|
||||
FROM schmelczera/great-ai:latest
|
||||
|
||||
# Remove this block if you don't have a requirements.txt
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --no-cache-dir --requirement requirements.txt
|
||||
|
||||
# If you store your models in S3 or GridFS, it may be a
|
||||
# good idea to cache them in the image so that you don't
|
||||
# have to download it each time a container starts
|
||||
RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor
|
||||
|
||||
# Add you application code to the image
|
||||
COPY . .
|
||||
|
||||
# The default ENTRYPOINT is great-ai, specify it's argument using CMD
|
||||
CMD ["deploy.ipynb"]
|
||||
|
||||
```
|
||||
|
||||
## Batch prediction
|
||||
|
||||
Processing larger amounts of data on a single machine is made easy by the [GreatAI][great_ai.GreatAI]'s [process_batch][great_ai.GreatAI.process_batch] method. This relies on multiprocessing ([parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map]) to take full advantage of all available CPU-cores.
|
||||
|
||||
```python
|
||||
>>> greeter.process_batch(['Alice', 'Bob'])
|
||||
[Trace[str]({'created': '2022-07-11T14:36:37.119183',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 5, 'arg:your_name:value': 'Alice'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.1251,
|
||||
'output': 'Hi Alice',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': '90ffa15f-e839-41c4-8e7a-3211168bc138'}),
|
||||
Trace[str]({'created': '2022-07-11T14:36:37.166659',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.0571,
|
||||
'output': 'Hi Bob',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': 'f48e94c7-0815-48b3-a864-41349d3dae84'})]
|
||||
```
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
# How to use a GreatAI service
|
||||
|
||||
After [creating a GreatAI service](/how-to-guides/cerate-service) by wrapping your prediction function, it's time to do some prediction.
|
||||
|
||||
Let's use the following example:
|
||||
|
||||
```python "type_safe_greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
||||
@GreatAI.create
|
||||
def type_safe_greeter(your_name: str) -> str:
|
||||
return f'Hi {your_name}'
|
||||
```
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue