Add more documentation
This commit is contained in:
parent
7165174f4f
commit
8d1ad9a242
21 changed files with 406 additions and 92 deletions
|
|
@ -207,10 +207,10 @@
|
|||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;226m2022-06-25 14:25:24,989 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,990 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongodbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,990 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongoDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,991 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising LargeFileMongo\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,992 | INFO | Settings: configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,993 | INFO | 🔩 tracing_database: MongodbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,993 | INFO | 🔩 tracing_database: MongoDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,994 | INFO | 🔩 large_file_implementation: LargeFileMongo\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,994 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:25:24,995 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
|
|
|
|||
|
|
@ -24,10 +24,10 @@
|
|||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;226m2022-06-25 14:57:20,629 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,629 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongodbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,629 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongoDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,630 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising LargeFileMongo\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,631 | INFO | Settings: configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,631 | INFO | 🔩 tracing_database: MongodbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,631 | INFO | 🔩 tracing_database: MongoDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,631 | INFO | 🔩 large_file_implementation: LargeFileMongo\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,632 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:57:20,633 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
|
|
|
|||
|
|
@ -33,10 +33,10 @@
|
|||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;226m2022-06-25 14:50:29,879 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,880 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongodbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,880 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongoDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,881 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising LargeFileMongo\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,881 | INFO | Settings: configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,882 | INFO | 🔩 tracing_database: MongodbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,882 | INFO | 🔩 tracing_database: MongoDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,883 | INFO | 🔩 large_file_implementation: LargeFileMongo\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,883 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-06-25 14:50:29,884 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
|
|
|
|||
|
|
@ -3,5 +3,5 @@
|
|||
A lot more details and discussion about the problem context and approaches of GreatAI can be found in my thesis.
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||
[::fontawesome-solid-graduation-cap: Download](thesis/main.py){ .md-button .md-button--primary }
|
||||
[::fontawesome-solid-graduation-cap: Download](thesis/main.pdf){ .md-button .md-button--primary }
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,2 @@
|
|||
# Call remote GreatAI instances
|
||||
|
||||
101
docs/how-to-guides/configure-service.md
Normal file
101
docs/how-to-guides/configure-service.md
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
# How to configure GreatAI
|
||||
|
||||
GreatAI aims to provide reasonable defaults wherever possible. The current configuration is always prominently displayed (and updated) on the dashboard and in the command-line startup banner.
|
||||
|
||||
## Using [great_ai.configure][]
|
||||
|
||||
You can override any of the default settings by calling [great_ai.configure][]. If you don't call `configure`, the default settings are applied on the first call to most `great-ai` functions.
|
||||
|
||||
!!! warning
|
||||
You must call [great_ai.configure][] before calling (or decorating with) any other `great-ai` function. However, importing other functions before calling [great_ai.configure][] is permitted.
|
||||
|
||||
```python title="configure-demo.py"
|
||||
from great_ai import configure, RouteConfig
|
||||
import logging
|
||||
|
||||
configure(
|
||||
version='1.0.0',
|
||||
log_level=logging.INFO,
|
||||
seed=2,
|
||||
should_log_exception_stack=False,
|
||||
prediction_cache_size=0, #(1)
|
||||
disable_se4ml_banner=True,
|
||||
dashboard_table_size=200,
|
||||
route_config=RouteConfig( #(2)
|
||||
feedback_endpoints_enabled=False,
|
||||
dashboard_enabled=False
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
1. Completely disable caching.
|
||||
2. The unspecified routes are enabled by default.
|
||||
|
||||
## Using remote storage
|
||||
|
||||
The only aspect that cannot be automated is choosing the backing storage for the database and file storage.
|
||||
|
||||
Right now, you have 3 options for storing the models and large datasets: [great_ai.large_file.LargeFileLocal][], [great_ai.large_file.LargeFileMongo][], and [great_ai.large_file.LargeFileS3][].
|
||||
|
||||
Without explicit configuration, [great_ai.large_file.LargeFileLocal][] is selected by default. This one still version-controls your files but it only stores them in a local path.
|
||||
|
||||
!!! important
|
||||
If your working directory contains a `mongo.ini` or `s3.ini` file, an attempt is made to auto-configure [LargeFileMongo][great_ai.large_file.LargeFileMongo] or [LargeFileS3][great_ai.large_file.LargeFileS3] respectively.
|
||||
|
||||
To use [LargeFileMongo][great_ai.large_file.LargeFileMongo] or [LargeFileS3][great_ai.large_file.LargeFileS3] explicitly, configure them before calling any other `great-ai` function.
|
||||
|
||||
### S3-compatible
|
||||
|
||||
```toml title="s3.ini"
|
||||
aws_region_name = eu-west-2
|
||||
aws_access_key_id = MY_AWS_ACCESS_KEY # ENV:MY_AWS_ACCESS_KEY would also work
|
||||
aws_secret_access_key = MY_AWS_SECRET_KEY
|
||||
large_files_bucket_name = bucket-for-models
|
||||
```
|
||||
|
||||
```python title="use-s3.py"
|
||||
from great_ai.large_file import LargeFileS3
|
||||
from great_ai import save_model
|
||||
|
||||
LargeFileS3.configure_credentials_from_file('s3.ini') #(1)
|
||||
|
||||
model = [4, 3]
|
||||
save_model(model, 'my-model')
|
||||
```
|
||||
|
||||
1. This line isn't strictly necceseary because if `s3.ini` (or `mongo.ini`) is available in the current working directory, they are automatically used to configure their respective LargeFile implementations/databases.
|
||||
|
||||
??? note "Departing from AWS"
|
||||
With the `aws_endpoint_url` argument, it is possible to use any other S3-compatible service such as [Backblaze](https://www.backblaze.com/){ target=_blank }. In that case, it would be `aws_endpoint_url=https://s3.us-west-002.backblazeb2.com`.
|
||||
|
||||
### GridFS
|
||||
|
||||
[GridFS](https://www.mongodb.com/docs/manual/core/gridfs/#:~:text=GridFS%20is%20a%20specification%20for,chunk%20as%20a%20separate%20document.){ target=_blank } specifies how to store files in MongoDB. The official MongoDB server and many compatible implementations support it.
|
||||
|
||||
```toml title="mongo.ini"
|
||||
MONGO_CONNECTION_STRING=mongodb://localhost:27017 # this is the default value
|
||||
# if `MONGO_CONNECTION_STRING` is specified, this default is overridden
|
||||
MONGO_CONNECTION_STRING=ENV:MONGO_CONNECTION_STRING
|
||||
MONGO_DATABASE=my-database # it is automatically created if doesn't exist
|
||||
```
|
||||
|
||||
```python title="use-mongo.py"
|
||||
from great_ai.large_file import LargeFileMongo
|
||||
from great_ai import save_model
|
||||
|
||||
LargeFileMongo.configure_credentials_from_file('mongo.ini')
|
||||
|
||||
model = [4, 3]
|
||||
save_model(model, 'my-model')
|
||||
```
|
||||
|
||||
!!! note "Simplifying config files"
|
||||
You can combine `mongo.ini` or `s3.ini` with your application's config file because the unneeded keys are ignored by the `configure_credentials_from_file` method.
|
||||
|
||||
## Using a database
|
||||
|
||||
By default, a thread-safe version of [TinyDB](https://tinydb.readthedocs.io/en/latest/){ target=_blank } is utilised for saving the prediction traces into a local file. Unfortunately, for most production needs, this method is not suitable.
|
||||
|
||||
### MongoDB
|
||||
|
||||
At the moment, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory, or explicitly call [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver.configure_credentials_from_file] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# How to create a GreatAI service
|
||||
|
||||
The core value of `great-ai` lies in its [GreatAI][great_ai.deploy.GreatAI] class. In order to take advantage of it, you need to create an instance wrapping your code.
|
||||
The core value of `great-ai` lies in its [GreatAI][great_ai.GreatAI] class. In order to take advantage of it, you need to create an instance wrapping your code.
|
||||
|
||||
Let's say that you have the following greeter function:
|
||||
|
||||
|
|
@ -9,7 +9,7 @@ def my_greeter_function(your_name):
|
|||
return f'Hi {your_name}!'
|
||||
```
|
||||
|
||||
You can simply decorate (wrap) this function using the [@GreatAI.create][great_ai.deploy.GreatAI.create] factory.
|
||||
You can simply decorate (wrap) this function using the [@GreatAI.create][great_ai.GreatAI.create] factory.
|
||||
|
||||
```python title="greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
|
@ -20,7 +20,7 @@ def greeter(your_name):
|
|||
```
|
||||
|
||||
??? info "Why not simply use `@GreatAI?`"
|
||||
The purpose of the [@GreatAI.create][great_ai.deploy.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
|
||||
The purpose of the [@GreatAI.create][great_ai.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
|
||||
|
||||
## With types
|
||||
|
||||
|
|
@ -52,9 +52,9 @@ async def async_greeter(your_name: str) -> str:
|
|||
|
||||
## With decorators
|
||||
|
||||
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.deploy.GreatAI.create] always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
|
||||
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.GreatAI.create] always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
|
||||
|
||||
### Using `use_model`
|
||||
### Using `@use_model`
|
||||
|
||||
If you have previously saved a model with [save_model][great_ai.save_model], you can inject it into your function by calling [@use_model][great_ai.use_model].
|
||||
|
||||
|
|
@ -72,14 +72,14 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
|
|||
1. By default, the parameter named `model` will be replaced by the loaded model. This behaviour can be customised by setting the `model_kwarg_name`. This way, even multiple models can be injected into a single function.
|
||||
|
||||
!!! important
|
||||
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.deploy.GreatAI.create]. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.GreatAI.create]. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
|
||||
### Using `parameter`
|
||||
### Using `@parameter`
|
||||
|
||||
If you wish to turn of logging or specify custom validation for your parameters, you can use the [@parameter][great_ai.parameter] decorator.
|
||||
|
||||
!!! note
|
||||
By default, all parameters that are not affected by an explicit [@parameter][great_ai.parameter] or [@use_model][great_ai.use_model] are automatically decorated with [@parameter][great_ai.parameter] when [@GreatAI.create][great_ai.deploy.GreatAI.create] is called.
|
||||
By default, all parameters that are not affected by an explicit [@parameter][great_ai.parameter] or [@use_model][great_ai.use_model] are automatically decorated with [@parameter][great_ai.parameter] when [@GreatAI.create][great_ai.GreatAI.create] is called.
|
||||
|
||||
```python title="greeter_with_validation.py"
|
||||
from great_ai import GreatAI, use_model
|
||||
|
|
@ -93,7 +93,7 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
|
|||
```
|
||||
|
||||
!!! important
|
||||
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.deploy.GreatAI.create]. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.GreatAI.create]. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
|
||||
## Complex example
|
||||
|
||||
|
|
|
|||
6
docs/how-to-guides/handle-training-data.md
Normal file
6
docs/how-to-guides/handle-training-data.md
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
# How to handle training data
|
||||
|
||||
## Upload data
|
||||
|
||||
## Use feedback
|
||||
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
# [open(S3)](https://pypi.org/project/open-large/)
|
||||
# How to use LargeFile
|
||||
|
||||
Storing, versioning, and downloading files from S3 made as easy as using `open()` in Python. Caching included.
|
||||
|
||||
|
|
@ -16,13 +16,88 @@
|
|||
") as client:\n",
|
||||
" pass"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | GreatAI (v0.1.3): configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 tracing_database: ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 large_file_implementation: LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 dashboard_table_size: 50\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | > Find out more at https://se-ml.github.io/practices\u001b[0m\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Trace[str]({'created': '2022-07-11T17:00:27.064568',\n",
|
||||
" 'exception': None,\n",
|
||||
" 'feedback': None,\n",
|
||||
" 'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},\n",
|
||||
" 'models': [],\n",
|
||||
" 'original_execution_time_ms': 0.0896,\n",
|
||||
" 'output': 'Hi Bob',\n",
|
||||
" 'tags': ['greeter', 'online', 'development'],\n",
|
||||
" 'trace_id': '8ff5b268-2613-4e85-96ae-f248666a051f'})"
|
||||
]
|
||||
},
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from great_ai import configure, RouteConfig\n",
|
||||
"from great_ai import GreatAI\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@GreatAI.create\n",
|
||||
"def greeter(your_name: str) -> str:\n",
|
||||
" return f\"Hi {your_name}\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"greeter(\"Bob\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.10.4 ('.env': venv)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.4"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "02dd6d3afbfa9fbbe1037d64ad9014965528a1ccad21929d6e72f466389a68ad"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
|
|
|
|||
131
docs/how-to-guides/use-service.md
Normal file
131
docs/how-to-guides/use-service.md
Normal file
|
|
@ -0,0 +1,131 @@
|
|||
# How to use a GreatAI service
|
||||
|
||||
After [creating a GreatAI service](/how-to-guides/create-service) by wrapping your prediction function, and optionally [configuring it](/how-to-guides/configure-service), it's time to do some prediction.
|
||||
|
||||
Let's take the following example:
|
||||
|
||||
```python title="greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
||||
@GreatAI.create
|
||||
def greeter(your_name: str) -> str:
|
||||
return f'Hi {your_name}'
|
||||
```
|
||||
|
||||
## One-off prediction
|
||||
|
||||
Even though `greeter` is now an instance of [GreatAI][great_ai.GreatAI], you can continue using it as a regular function.
|
||||
|
||||
```python
|
||||
>>> greeter('Bob')
|
||||
Trace[str]({'created': '2022-07-11T14:31:46.183764',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.0381,
|
||||
'output': 'Hi Bob',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': '7c284fd7-7f0d-4464-b5f8-3ef126df34af'})
|
||||
```
|
||||
|
||||
As you can see, the original return value is wrapped in a [Trace][great_ai.Trace] object (which is also persisted in your database of choice). You can access the original value under the `output` property.
|
||||
|
||||
## Online prediction
|
||||
|
||||
Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
|
||||
|
||||
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To this end, `uvicorn` is wrapped by the `great-ai` command-line utility which, among others, takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
|
||||
|
||||
### In development
|
||||
|
||||
```sh
|
||||
great-ai greeter.py
|
||||
```
|
||||
|
||||
!!! success
|
||||
Your model is accessible at [localhost:6060](http:/127.0.0.1:6060){ target=_blank }.
|
||||
|
||||
Some configuration options are also supported.
|
||||
|
||||
```sh
|
||||
great-ai greeter.py --port 8000 --host 127.0.0.1 --timeout_keep_alive 10
|
||||
```
|
||||
> For more options (but no Notebook support, use [uvicorn](https://www.uvicorn.org/){ target=_blank })
|
||||
|
||||
### In production
|
||||
|
||||
There are three main approaches for deploying a GreatAI service.
|
||||
|
||||
#### Manual deployment
|
||||
|
||||
The app is run in *production-mode* if the value of the `ENVIRONMENT` environment variable is set to `production`.
|
||||
|
||||
```sh
|
||||
ENVIRONMENT=production great-ai greeter.py
|
||||
```
|
||||
|
||||
Simply run `ENVIRONMENT=production great-ai deploy.ipynb` in the command-line of a production machine.
|
||||
> This is the crudest approach, however, it might be fitting for some contexts.
|
||||
|
||||
#### Containerised deployment
|
||||
|
||||
Run the notebook directly in a container or create a service for it using your favourite container orchestrator.
|
||||
|
||||
```sh
|
||||
docker run -p 6060:6060 --volume `pwd`:/app --rm \
|
||||
schmelczera/great-ai deploy.ipynb
|
||||
```
|
||||
> You can replace ``pwd`` with the path to your code's folder.
|
||||
|
||||
#### Use a Platform-as-a-Service
|
||||
|
||||
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank } or [MLEM](https://mlem.ai/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
|
||||
|
||||
To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.
|
||||
|
||||
```Dockerfile
|
||||
FROM schmelczera/great-ai:latest
|
||||
|
||||
# Remove this block if you don't have a requirements.txt
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --no-cache-dir --requirement requirements.txt
|
||||
|
||||
# If you store your models in S3 or GridFS, it may be a
|
||||
# good idea to cache them in the image so that you don't
|
||||
# have to download it each time a container starts
|
||||
RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor
|
||||
|
||||
# Add you application code to the image
|
||||
COPY . .
|
||||
|
||||
# The default ENTRYPOINT is great-ai, specify it's argument using CMD
|
||||
CMD ["deploy.ipynb"]
|
||||
|
||||
```
|
||||
|
||||
## Batch prediction
|
||||
|
||||
Processing larger amounts of data on a single machine is made easy by the [GreatAI][great_ai.GreatAI]'s [process_batch][great_ai.GreatAI.process_batch] method. This relies on multiprocessing ([parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map]) to take full advantage of all available CPU-cores.
|
||||
|
||||
```python
|
||||
>>> greeter.process_batch(['Alice', 'Bob'])
|
||||
[Trace[str]({'created': '2022-07-11T14:36:37.119183',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 5, 'arg:your_name:value': 'Alice'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.1251,
|
||||
'output': 'Hi Alice',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': '90ffa15f-e839-41c4-8e7a-3211168bc138'}),
|
||||
Trace[str]({'created': '2022-07-11T14:36:37.166659',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.0571,
|
||||
'output': 'Hi Bob',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': 'f48e94c7-0815-48b3-a864-41349d3dae84'})]
|
||||
```
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
# How to use a GreatAI service
|
||||
|
||||
After [creating a GreatAI service](/how-to-guides/cerate-service) by wrapping your prediction function, it's time to do some prediction.
|
||||
|
||||
Let's use the following example:
|
||||
|
||||
```python "type_safe_greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
||||
@GreatAI.create
|
||||
def type_safe_greeter(your_name: str) -> str:
|
||||
return f'Hi {your_name}'
|
||||
```
|
||||
|
||||
|
|
@ -111,5 +111,5 @@ However, research indicates that professionals rarely use them. This may be due
|
|||
|
||||
GreatAI has been battle-tested on the core platform services of [ScoutinScience](https://www.scoutinscience.com/){ target=_blank }.
|
||||
|
||||
|
||||
|
||||
{ loading=lazy }
|
||||
{ loading=lazy }
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Reference
|
||||
# GreatAI reference
|
||||
|
||||
```python
|
||||
from great_ai import *
|
||||
|
|
@ -54,3 +54,17 @@ from great_ai import *
|
|||
::: great_ai.delete_ground_truth
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
||||
## Tracing databases
|
||||
|
||||
::: great_ai.TracingDatabaseDriver
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
||||
::: great_ai.MongoDbDriver
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
||||
::: great_ai.ParallelTinyDbDriver
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
|
|
|||
|
|
@ -3,3 +3,19 @@
|
|||
```python
|
||||
from great_ai.large_file import *
|
||||
```
|
||||
|
||||
::: great_ai.large_file.LargeFileLocal
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
||||
::: great_ai.large_file.LargeFileS3
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
||||
::: great_ai.large_file.LargeFileMongo
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
||||
::: great_ai.large_file.LargeFileBase
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
|
|
|||
|
|
@ -10,22 +10,26 @@ Well-tested tools that can be used in production with confidence. The toolbox of
|
|||
|
||||
::: great_ai.utilities.clean
|
||||
::: great_ai.utilities.get_sentences
|
||||
::: great_ai.utilities.language.predict_language
|
||||
::: great_ai.utilities.language.is_english
|
||||
::: great_ai.utilities.language.english_name_of_language
|
||||
::: great_ai.utilities.evaluate_ranking.evaluate_ranking
|
||||
::: great_ai.utilities.predict_language
|
||||
::: great_ai.utilities.is_english
|
||||
::: great_ai.utilities.english_name_of_language
|
||||
::: great_ai.utilities.evaluate_ranking
|
||||
|
||||
## Parallel processing
|
||||
|
||||
Multiprocessing and multithreading-based parallelism with support for `async` functions. Its main purpose is to implement [great_ai.GreatAI.process_batch][], however, the parallel processing functions are also convenient for covering other types of mapping needs with a friendlier API than [joblib](https://joblib.readthedocs.io/en/latest/parallel.html){ target=_blank } or [multiprocess](https://pypi.org/project/multiprocess/){ target=_blank }.
|
||||
|
||||
::: great_ai.utilities.parallel_map.simple_parallel_map
|
||||
::: great_ai.utilities.simple_parallel_map
|
||||
options:
|
||||
show_root_heading: true
|
||||
::: great_ai.utilities.parallel_map.parallel_map
|
||||
::: great_ai.utilities.parallel_map.threaded_parallel_map
|
||||
::: great_ai.utilities.threaded_parallel_map
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
||||
## Composable parallel processing
|
||||
|
||||
Because both [threaded_parallel_map][great_ai.utilities.parallel_map.threaded_parallel_map.threaded_parallel_map] and [parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map] have a streaming interface, it is easy to compose them and end up with, for example, a process for each CPU core with its own thread-pool or event-loop. Longer pipelines are also easy to imagine. The chunking methods help in these compositions.
|
||||
Because both [threaded_parallel_map][great_ai.utilities.parallel_map.threaded_parallel_map.threaded_parallel_map] and [parallel_map][great_ai.utilities.parallel_map.parallel_map] have a streaming interface, it is easy to compose them and end up with, for example, a process for each CPU core with its own thread-pool or event-loop. Longer pipelines are also easy to imagine. The chunking methods help in these compositions.
|
||||
|
||||
For more info, check-out [the scraping guide](/how-to-guides/scraping).
|
||||
|
||||
|
|
@ -34,5 +38,10 @@ For more info, check-out [the scraping guide](/how-to-guides/scraping).
|
|||
|
||||
## Operations
|
||||
|
||||
::: great_ai.utilities.config_file.config_file
|
||||
::: great_ai.utilities.logger.get_logger
|
||||
::: great_ai.utilities.ConfigFile
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
||||
::: great_ai.utilities.get_logger
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
# View models
|
||||
|
||||
::: great_ai.views.trace.Trace
|
||||
::: great_ai.Trace
|
||||
options:
|
||||
show_root_heading: true
|
||||
|
|
|
|||
|
|
@ -131,49 +131,7 @@
|
|||
"source": [
|
||||
"Now that you've made sure your application is hardened enough for the intended use case, it is time to deploy it. The responsibilities of GreatAI end when it wraps your inference function and model into a production-ready service. You're given the freedom and responsibility to deploy this service. Fortunately, you (or your organisation) probably already has an established routine for deploying services.\n",
|
||||
"\n",
|
||||
"There are three main approaches to deploy a GreatAI service.\n",
|
||||
"\n",
|
||||
"### Manual deployment\n",
|
||||
"\n",
|
||||
"Simply run `ENVIRONMENT=production great-ai deploy.ipynb` in the command-line of a production machine.\n",
|
||||
"> This is the crudest approach, however, it might be fitting for some contexts.\n",
|
||||
"\n",
|
||||
"### Containerised deployment\n",
|
||||
"\n",
|
||||
"Run the notebook directly in a container or create a service for it using your favourite orchestrator.\n",
|
||||
"\n",
|
||||
"```sh\n",
|
||||
"docker run \\\n",
|
||||
" -p 6060:6060 \\\n",
|
||||
" --volume `pwd`:/app \\\n",
|
||||
" --rm \\\n",
|
||||
" schmelczera/great-ai deploy.ipynb\n",
|
||||
"```\n",
|
||||
"> You can replace ``pwd`` with the path to your code's folder.\n",
|
||||
"\n",
|
||||
"#### Use a Platform-as-a-Service\n",
|
||||
"\n",
|
||||
"Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers that take a Docker image as a source and handle the rest of the deployment.\n",
|
||||
"\n",
|
||||
"To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as pytorch or tensorflow.\n",
|
||||
"\n",
|
||||
"```Dockerfile\n",
|
||||
"FROM schmelczera/great-ai:latest\n",
|
||||
"\n",
|
||||
"# Remove this block if you don't have a requirements.txt\n",
|
||||
"COPY requirements.txt ./ \n",
|
||||
"RUN pip install --no-cache-dir --requirement requirements.txt\n",
|
||||
"\n",
|
||||
"# If you store your models in S3 or GridFS, it may be a \n",
|
||||
"# good idea to # cache them in the image so that you don't\n",
|
||||
"# have to download it each time a container starts\n",
|
||||
"# RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor\n",
|
||||
"\n",
|
||||
"COPY . .\n",
|
||||
"\n",
|
||||
"CMD [\"deploy.ipynb\"]\n",
|
||||
"\n",
|
||||
"```"
|
||||
"There are three main approaches to deploy a GreatAI service: For more info about them, check out [the deployment how-to](/how-to-guides/use-service)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -70,4 +70,4 @@ def predict_domain(sentence, model):
|
|||
1. [@use_model][great_ai.use_model] loads and injects your model into the `predict_domain` function's `model` argument.
|
||||
You can freely reference it knowing that it is always given to the function.
|
||||
|
||||
Finally, we deploy the model, inference function, and the GreatAI wrapping all of these. For that we either use: `great-ai deploy.ipynb` or build a Docker image.
|
||||
Finally, we test the model's inference function through the GreatAI dashboard. [The only thing left is to deploy the hardened-service.](/how-to-guides/use-service)
|
||||
16
mkdocs.yaml
16
mkdocs.yaml
|
|
@ -94,10 +94,24 @@ nav:
|
|||
- tutorial/deploy.ipynb
|
||||
- User Guides:
|
||||
- how-to-guides/create-service.md
|
||||
- how-to-guides/configure-service.md
|
||||
- how-to-guides/use-service.md
|
||||
- how-to-guides/handle-training-data.md
|
||||
- how-to-guides/large_file.md
|
||||
- how-to-guides/call_remote.md
|
||||
- how-to-guides/scraping.ipynb
|
||||
- Reference:
|
||||
- reference/index.md
|
||||
- reference/utilities.md
|
||||
- reference/large-file.md
|
||||
- reference/views.md
|
||||
- Examples:
|
||||
- Explainable Naive Bayes:
|
||||
- examples/simple/data.ipynb
|
||||
- examples/simple/train.ipynb
|
||||
- examples/simple/deploy.ipynb
|
||||
# - Explainable SciBERT:
|
||||
# - examples/scibert/data.ipynb
|
||||
# - examples/scibert/train.ipynb
|
||||
# - examples/scibert/deploy.ipynb
|
||||
- explanation.md
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue