Deployed 8d1ad9a with MkDocs version: 1.3.0
This commit is contained in:
parent
1a6a68ffcc
commit
25a66fb86b
46 changed files with 10768 additions and 1924 deletions
|
|
@ -0,0 +1,2 @@
|
|||
# Call remote GreatAI instances
|
||||
|
||||
|
|
@ -17,7 +17,7 @@
|
|||
|
||||
|
||||
|
||||
<title>Call remote - GreatAI documentation</title>
|
||||
<title>Call remote GreatAI instances - GreatAI documentation</title>
|
||||
|
||||
|
||||
|
||||
|
|
@ -69,6 +69,11 @@
|
|||
<label class="md-overlay" for="__drawer"></label>
|
||||
<div data-md-component="skip">
|
||||
|
||||
|
||||
<a href="#call-remote-greatai-instances" class="md-skip">
|
||||
Skip to content
|
||||
</a>
|
||||
|
||||
</div>
|
||||
<div data-md-component="announce">
|
||||
|
||||
|
|
@ -98,7 +103,7 @@
|
|||
<div class="md-header__topic" data-md-component="header-topic">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Call remote
|
||||
Call remote GreatAI instances
|
||||
|
||||
</span>
|
||||
</div>
|
||||
|
|
@ -355,6 +360,90 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../configure-service/" class="md-nav__link">
|
||||
How to configure GreatAI
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../use-service/" class="md-nav__link">
|
||||
How to use a GreatAI service
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../handle-training-data/" class="md-nav__link">
|
||||
How to handle training data
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../large_file/" class="md-nav__link">
|
||||
How to use LargeFile
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../call_remote.md" class="md-nav__link">
|
||||
None
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../scraping/" class="md-nav__link">
|
||||
Scraping
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
|
@ -396,7 +485,7 @@
|
|||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
GreatAI reference
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
|
@ -458,6 +547,115 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5" type="checkbox" id="__nav_5" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_5">
|
||||
Examples
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Examples" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_5">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Examples
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5_1" type="checkbox" id="__nav_5_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_5_1">
|
||||
Explainable Naive Bayes
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Explainable Naive Bayes" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_5_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Explainable Naive Bayes
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
|
|
@ -483,6 +681,8 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -503,16 +703,14 @@
|
|||
|
||||
|
||||
|
||||
<h1>Call remote</h1>
|
||||
|
||||
|
||||
<h1 id="call-remote-greatai-instances">Call remote GreatAI instances<a class="headerlink" href="#call-remote-greatai-instances" title="Permanent link">#</a></h1>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 10, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 11, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
101
how-to-guides/configure-service/configure-service.md
Normal file
101
how-to-guides/configure-service/configure-service.md
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
# How to configure GreatAI
|
||||
|
||||
GreatAI aims to provide reasonable defaults wherever possible. The current configuration is always prominently displayed (and updated) on the dashboard and in the command-line startup banner.
|
||||
|
||||
## Using [great_ai.configure][]
|
||||
|
||||
You can override any of the default settings by calling [great_ai.configure][]. If you don't call `configure`, the default settings are applied on the first call to most `great-ai` functions.
|
||||
|
||||
!!! warning
|
||||
You must call [great_ai.configure][] before calling (or decorating with) any other `great-ai` function. However, importing other functions before calling [great_ai.configure][] is permitted.
|
||||
|
||||
```python title="configure-demo.py"
|
||||
from great_ai import configure, RouteConfig
|
||||
import logging
|
||||
|
||||
configure(
|
||||
version='1.0.0',
|
||||
log_level=logging.INFO,
|
||||
seed=2,
|
||||
should_log_exception_stack=False,
|
||||
prediction_cache_size=0, #(1)
|
||||
disable_se4ml_banner=True,
|
||||
dashboard_table_size=200,
|
||||
route_config=RouteConfig( #(2)
|
||||
feedback_endpoints_enabled=False,
|
||||
dashboard_enabled=False
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
1. Completely disable caching.
|
||||
2. The unspecified routes are enabled by default.
|
||||
|
||||
## Using remote storage
|
||||
|
||||
The only aspect that cannot be automated is choosing the backing storage for the database and file storage.
|
||||
|
||||
Right now, you have 3 options for storing the models and large datasets: [great_ai.large_file.LargeFileLocal][], [great_ai.large_file.LargeFileMongo][], and [great_ai.large_file.LargeFileS3][].
|
||||
|
||||
Without explicit configuration, [great_ai.large_file.LargeFileLocal][] is selected by default. This one still version-controls your files but it only stores them in a local path.
|
||||
|
||||
!!! important
|
||||
If your working directory contains a `mongo.ini` or `s3.ini` file, an attempt is made to auto-configure [LargeFileMongo][great_ai.large_file.LargeFileMongo] or [LargeFileS3][great_ai.large_file.LargeFileS3] respectively.
|
||||
|
||||
To use [LargeFileMongo][great_ai.large_file.LargeFileMongo] or [LargeFileS3][great_ai.large_file.LargeFileS3] explicitly, configure them before calling any other `great-ai` function.
|
||||
|
||||
### S3-compatible
|
||||
|
||||
```toml title="s3.ini"
|
||||
aws_region_name = eu-west-2
|
||||
aws_access_key_id = MY_AWS_ACCESS_KEY # ENV:MY_AWS_ACCESS_KEY would also work
|
||||
aws_secret_access_key = MY_AWS_SECRET_KEY
|
||||
large_files_bucket_name = bucket-for-models
|
||||
```
|
||||
|
||||
```python title="use-s3.py"
|
||||
from great_ai.large_file import LargeFileS3
|
||||
from great_ai import save_model
|
||||
|
||||
LargeFileS3.configure_credentials_from_file('s3.ini') #(1)
|
||||
|
||||
model = [4, 3]
|
||||
save_model(model, 'my-model')
|
||||
```
|
||||
|
||||
1. This line isn't strictly necceseary because if `s3.ini` (or `mongo.ini`) is available in the current working directory, they are automatically used to configure their respective LargeFile implementations/databases.
|
||||
|
||||
??? note "Departing from AWS"
|
||||
With the `aws_endpoint_url` argument, it is possible to use any other S3-compatible service such as [Backblaze](https://www.backblaze.com/){ target=_blank }. In that case, it would be `aws_endpoint_url=https://s3.us-west-002.backblazeb2.com`.
|
||||
|
||||
### GridFS
|
||||
|
||||
[GridFS](https://www.mongodb.com/docs/manual/core/gridfs/#:~:text=GridFS%20is%20a%20specification%20for,chunk%20as%20a%20separate%20document.){ target=_blank } specifies how to store files in MongoDB. The official MongoDB server and many compatible implementations support it.
|
||||
|
||||
```toml title="mongo.ini"
|
||||
MONGO_CONNECTION_STRING=mongodb://localhost:27017 # this is the default value
|
||||
# if `MONGO_CONNECTION_STRING` is specified, this default is overridden
|
||||
MONGO_CONNECTION_STRING=ENV:MONGO_CONNECTION_STRING
|
||||
MONGO_DATABASE=my-database # it is automatically created if doesn't exist
|
||||
```
|
||||
|
||||
```python title="use-mongo.py"
|
||||
from great_ai.large_file import LargeFileMongo
|
||||
from great_ai import save_model
|
||||
|
||||
LargeFileMongo.configure_credentials_from_file('mongo.ini')
|
||||
|
||||
model = [4, 3]
|
||||
save_model(model, 'my-model')
|
||||
```
|
||||
|
||||
!!! note "Simplifying config files"
|
||||
You can combine `mongo.ini` or `s3.ini` with your application's config file because the unneeded keys are ignored by the `configure_credentials_from_file` method.
|
||||
|
||||
## Using a database
|
||||
|
||||
By default, a thread-safe version of [TinyDB](https://tinydb.readthedocs.io/en/latest/){ target=_blank } is utilised for saving the prediction traces into a local file. Unfortunately, for most production needs, this method is not suitable.
|
||||
|
||||
### MongoDB
|
||||
|
||||
At the moment, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory, or explicitly call [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver.configure_credentials_from_file] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].
|
||||
1053
how-to-guides/configure-service/index.html
Normal file
1053
how-to-guides/configure-service/index.html
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -1,6 +1,6 @@
|
|||
# How to create a GreatAI service
|
||||
|
||||
The core value of `great-ai` lies in its [GreatAI][great_ai.deploy.GreatAI] class. In order to take advantage of it, you need to create an instance wrapping your code.
|
||||
The core value of `great-ai` lies in its [GreatAI][great_ai.GreatAI] class. In order to take advantage of it, you need to create an instance wrapping your code.
|
||||
|
||||
Let's say that you have the following greeter function:
|
||||
|
||||
|
|
@ -9,7 +9,7 @@ def my_greeter_function(your_name):
|
|||
return f'Hi {your_name}!'
|
||||
```
|
||||
|
||||
You can simply decorate (wrap) this function using the [@GreatAI.create][great_ai.deploy.GreatAI.create] factory.
|
||||
You can simply decorate (wrap) this function using the [@GreatAI.create][great_ai.GreatAI.create] factory.
|
||||
|
||||
```python title="greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
|
@ -20,7 +20,7 @@ def greeter(your_name):
|
|||
```
|
||||
|
||||
??? info "Why not simply use `@GreatAI?`"
|
||||
The purpose of the [@GreatAI.create][great_ai.deploy.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
|
||||
The purpose of the [@GreatAI.create][great_ai.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
|
||||
|
||||
## With types
|
||||
|
||||
|
|
@ -52,9 +52,9 @@ async def async_greeter(your_name: str) -> str:
|
|||
|
||||
## With decorators
|
||||
|
||||
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.deploy.GreatAI.create] always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
|
||||
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.GreatAI.create] always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
|
||||
|
||||
### Using `use_model`
|
||||
### Using `@use_model`
|
||||
|
||||
If you have previously saved a model with [save_model][great_ai.save_model], you can inject it into your function by calling [@use_model][great_ai.use_model].
|
||||
|
||||
|
|
@ -72,14 +72,14 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
|
|||
1. By default, the parameter named `model` will be replaced by the loaded model. This behaviour can be customised by setting the `model_kwarg_name`. This way, even multiple models can be injected into a single function.
|
||||
|
||||
!!! important
|
||||
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.deploy.GreatAI.create]. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.GreatAI.create]. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
|
||||
### Using `parameter`
|
||||
### Using `@parameter`
|
||||
|
||||
If you wish to turn of logging or specify custom validation for your parameters, you can use the [@parameter][great_ai.parameter] decorator.
|
||||
|
||||
!!! note
|
||||
By default, all parameters that are not affected by an explicit [@parameter][great_ai.parameter] or [@use_model][great_ai.use_model] are automatically decorated with [@parameter][great_ai.parameter] when [@GreatAI.create][great_ai.deploy.GreatAI.create] is called.
|
||||
By default, all parameters that are not affected by an explicit [@parameter][great_ai.parameter] or [@use_model][great_ai.use_model] are automatically decorated with [@parameter][great_ai.parameter] when [@GreatAI.create][great_ai.GreatAI.create] is called.
|
||||
|
||||
```python title="greeter_with_validation.py"
|
||||
from great_ai import GreatAI, use_model
|
||||
|
|
@ -93,7 +93,7 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
|
|||
```
|
||||
|
||||
!!! important
|
||||
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.deploy.GreatAI.create]. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.GreatAI.create]. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
|
||||
## Complex example
|
||||
|
||||
|
|
|
|||
|
|
@ -411,14 +411,14 @@
|
|||
|
||||
<li class="md-nav__item">
|
||||
<a href="#using-use_model" class="md-nav__link">
|
||||
Using use_model
|
||||
Using @use_model
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#using-parameter" class="md-nav__link">
|
||||
Using parameter
|
||||
Using @parameter
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
|
@ -444,6 +444,90 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../configure-service/" class="md-nav__link">
|
||||
How to configure GreatAI
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../use-service/" class="md-nav__link">
|
||||
How to use a GreatAI service
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../handle-training-data/" class="md-nav__link">
|
||||
How to handle training data
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../large_file/" class="md-nav__link">
|
||||
How to use LargeFile
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../call_remote.md" class="md-nav__link">
|
||||
None
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../scraping/" class="md-nav__link">
|
||||
Scraping
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
|
@ -485,7 +569,7 @@
|
|||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
GreatAI reference
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
|
@ -547,6 +631,115 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5" type="checkbox" id="__nav_5" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_5">
|
||||
Examples
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Examples" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_5">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Examples
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5_1" type="checkbox" id="__nav_5_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_5_1">
|
||||
Explainable Naive Bayes
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Explainable Naive Bayes" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_5_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Explainable Naive Bayes
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
|
|
@ -605,14 +798,14 @@
|
|||
|
||||
<li class="md-nav__item">
|
||||
<a href="#using-use_model" class="md-nav__link">
|
||||
Using use_model
|
||||
Using @use_model
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#using-parameter" class="md-nav__link">
|
||||
Using parameter
|
||||
Using @parameter
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
|
@ -688,7 +881,7 @@
|
|||
</code></pre></div>
|
||||
<h2 id="with-decorators">With decorators<a class="headerlink" href="#with-decorators" title="Permanent link">#</a></h2>
|
||||
<p>GreatAI can decorate already decorated functions. The only restriction is that <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.</p>
|
||||
<h3 id="using-use_model">Using <code>use_model</code><a class="headerlink" href="#using-use_model" title="Permanent link">#</a></h3>
|
||||
<h3 id="using-use_model">Using <code>@use_model</code><a class="headerlink" href="#using-use_model" title="Permanent link">#</a></h3>
|
||||
<p>If you have previously saved a model with <a class="autorefs autorefs-internal" href="../../reference/#great_ai.save_model">save_model</a>, you can inject it into your function by calling <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a>.</p>
|
||||
<div class="highlight"><span class="filename">greeter_with_model.py</span><pre><span></span><code><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span><span class="p">,</span> <span class="n">use_model</span>
|
||||
<a id="__codelineno-4-2" name="__codelineno-4-2" href="#__codelineno-4-2"></a>
|
||||
|
|
@ -706,7 +899,7 @@
|
|||
<p class="admonition-title">Important</p>
|
||||
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> in other places of the codebase, it works equally well outside of GreatAI services. </p>
|
||||
</div>
|
||||
<h3 id="using-parameter">Using <code>parameter</code><a class="headerlink" href="#using-parameter" title="Permanent link">#</a></h3>
|
||||
<h3 id="using-parameter">Using <code>@parameter</code><a class="headerlink" href="#using-parameter" title="Permanent link">#</a></h3>
|
||||
<p>If you wish to turn of logging or specify custom validation for your parameters, you can use the <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> decorator.</p>
|
||||
<div class="admonition note">
|
||||
<p class="admonition-title">Note</p>
|
||||
|
|
@ -785,13 +978,13 @@
|
|||
|
||||
|
||||
|
||||
<a href="../../reference/" class="md-footer__link md-footer__link--next" aria-label="Next: Reference" rel="next">
|
||||
<a href="../configure-service/" class="md-footer__link md-footer__link--next" aria-label="Next: How to configure GreatAI" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
Reference
|
||||
How to configure GreatAI
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
|
|
|
|||
|
|
@ -0,0 +1,6 @@
|
|||
# How to handle training data
|
||||
|
||||
## Upload data
|
||||
|
||||
## Use feedback
|
||||
|
||||
|
|
@ -17,7 +17,7 @@
|
|||
|
||||
|
||||
|
||||
<title>How to use a GreatAI service - GreatAI documentation</title>
|
||||
<title>How to handle training data - GreatAI documentation</title>
|
||||
|
||||
|
||||
|
||||
|
|
@ -70,7 +70,7 @@
|
|||
<div data-md-component="skip">
|
||||
|
||||
|
||||
<a href="#how-to-use-a-greatai-service" class="md-skip">
|
||||
<a href="#how-to-handle-training-data" class="md-skip">
|
||||
Skip to content
|
||||
</a>
|
||||
|
||||
|
|
@ -103,7 +103,7 @@
|
|||
<div class="md-header__topic" data-md-component="header-topic">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
How to use a GreatAI service
|
||||
How to handle training data
|
||||
|
||||
</span>
|
||||
</div>
|
||||
|
|
@ -324,12 +324,14 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" checked>
|
||||
|
||||
|
||||
|
||||
|
|
@ -360,6 +362,138 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../configure-service/" class="md-nav__link">
|
||||
How to configure GreatAI
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../use-service/" class="md-nav__link">
|
||||
How to use a GreatAI service
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link md-nav__link--active" for="__toc">
|
||||
How to handle training data
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
How to handle training data
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Table of contents
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#upload-data" class="md-nav__link">
|
||||
Upload data
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#use-feedback" class="md-nav__link">
|
||||
Use feedback
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../large_file/" class="md-nav__link">
|
||||
How to use LargeFile
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../call_remote.md" class="md-nav__link">
|
||||
None
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../scraping/" class="md-nav__link">
|
||||
Scraping
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
|
@ -401,7 +535,7 @@
|
|||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
GreatAI reference
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
|
@ -463,6 +597,115 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5" type="checkbox" id="__nav_5" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_5">
|
||||
Examples
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Examples" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_5">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Examples
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5_1" type="checkbox" id="__nav_5_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_5_1">
|
||||
Explainable Naive Bayes
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Explainable Naive Bayes" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_5_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Explainable Naive Bayes
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
|
|
@ -491,6 +734,28 @@
|
|||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Table of contents
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#upload-data" class="md-nav__link">
|
||||
Upload data
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#use-feedback" class="md-nav__link">
|
||||
Use feedback
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -504,28 +769,22 @@
|
|||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/using-service.md" title="Edit this page" class="md-content__button md-icon">
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/handle-training-data.md" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<h1 id="how-to-use-a-greatai-service">How to use a GreatAI service<a class="headerlink" href="#how-to-use-a-greatai-service" title="Permanent link">#</a></h1>
|
||||
<p>After <a href="/how-to-guides/cerate-service">creating a GreatAI service</a> by wrapping your prediction function, it's time to do some prediction.</p>
|
||||
<p>Let's use the following example:</p>
|
||||
<p>```python "type_safe_greeter.py"
|
||||
from great_ai import GreatAI</p>
|
||||
<p>@GreatAI.create
|
||||
def type_safe_greeter(your_name: str) -> str:
|
||||
return f'Hi {your_name}'
|
||||
```</p>
|
||||
<h1 id="how-to-handle-training-data">How to handle training data<a class="headerlink" href="#how-to-handle-training-data" title="Permanent link">#</a></h1>
|
||||
<h2 id="upload-data">Upload data<a class="headerlink" href="#upload-data" title="Permanent link">#</a></h2>
|
||||
<h2 id="use-feedback">Use feedback<a class="headerlink" href="#use-feedback" title="Permanent link">#</a></h2>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 10, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 11, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -542,6 +801,42 @@ def type_safe_greeter(your_name: str) -> str:
|
|||
|
||||
<footer class="md-footer">
|
||||
|
||||
|
||||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
<a href="../use-service/" class="md-footer__link md-footer__link--prev" aria-label="Previous: How to use a GreatAI service" rel="prev">
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</div>
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Previous
|
||||
</span>
|
||||
How to use a GreatAI service
|
||||
</div>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="../large_file/" class="md-footer__link md-footer__link--next" aria-label="Next: How to use LargeFile" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
How to use LargeFile
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
</nav>
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
1136
how-to-guides/large_file/index.html
Normal file
1136
how-to-guides/large_file/index.html
Normal file
File diff suppressed because it is too large
Load diff
118
how-to-guides/large_file/large_file.md
Normal file
118
how-to-guides/large_file/large_file.md
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
# How to use LargeFile
|
||||
|
||||
Storing, versioning, and downloading files from S3 made as easy as using `open()` in Python. Caching included.
|
||||
|
||||
## Motivation
|
||||
|
||||
Oftentimes, especially when working with data-heavy applications, large files can proliferate in a repository. Version controlling them is an obvious next step, however, GitHub's git LFS implementation [doesn't support the deletion](https://docs.github.com/en/repositories/working-with-files/managing-large-files/removing-files-from-git-large-file-storage#git-lfs-objects-in-your-repository) of large files, making it easy for them to eat-up the LFS quota and explode the size of your repos.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
pip install open-large
|
||||
```
|
||||
|
||||
### Simple example
|
||||
|
||||
```python
|
||||
from great_ai.large_file import LargeFileS3
|
||||
|
||||
LargeFileS3.configure_credentials({
|
||||
"aws_region_name": "your_region_like_eu-west-2",
|
||||
"aws_access_key_id": "YOUR_ACCESS_KEY_ID",
|
||||
"aws_secret_access_key": "YOUR_VERY_SECRET_ACCESS_KEY",
|
||||
"large_files_bucket_name": "create_a_bucket_and_put_its_name_here",
|
||||
})
|
||||
|
||||
# Creates a new version and deletes the older version leaving the 3 most recently used intact
|
||||
with LargeFileS3("test.txt", "w", keep_last_n=3) as f:
|
||||
for i in range(100000):
|
||||
f.write('test\n')
|
||||
|
||||
# By default the latest version is returned
|
||||
# but an optional `version` keyword argument can be provided as well
|
||||
with LargeFileS3("test.txt", "r") as f:
|
||||
print(f.readlines()[0])
|
||||
```
|
||||
|
||||
> Automatically creates a file, writes to it, uploads it to S3, and then queries the most recent version of it.
|
||||
> In this case, the latest version is already in the local cache, no download is required.
|
||||
|
||||
### More details
|
||||
|
||||
`LargeFile` behaves like an opened file (in the background it is a temp file after all). Binary reading and writing is supported along with the [different keywords](https://docs.python.org/3/library/functions.html#open) `open()` accepts.
|
||||
|
||||
The local cache can be configured with these properties:
|
||||
|
||||
```python
|
||||
LargeFile.cache_path = Path('.cache')
|
||||
LargeFile.max_cache_size = "30 GB"
|
||||
```
|
||||
|
||||
#### I only need a path
|
||||
|
||||
In case you only need a path to the "remote" file, this pattern can be applied:
|
||||
|
||||
```python
|
||||
path_to_model = LargeFile("folder-of-my-bert-model", version=31).get()
|
||||
```
|
||||
|
||||
> This will first download the file/folder into your local cache folder. Then, it returns a `Path` object to the local version. Which can be turned into a string with `str(path_to_model)`.
|
||||
|
||||
The same approach works for uploads:
|
||||
|
||||
```python
|
||||
LargeFile("folder-of-my-bert-model").push('path_to_local/folder_or_file')
|
||||
```
|
||||
|
||||
> This way, both regular files and folders can be handled. The uploaded file is called **folder-of-my-bert-model**, the local name is ignored.
|
||||
|
||||
Lastly, all version of the remote object can be deleted by calling `LargeFile("my-file").delete()`. It will still reside in your local cache afterwards, its deletion will happen next time your local cache has to be pruned.
|
||||
|
||||
### Command-line example
|
||||
|
||||
The package can be used as a module from the command-line to give you more flexibility.
|
||||
|
||||
#### Setup
|
||||
|
||||
Create an .ini file (or use _~/.aws/credentials_). It may look like this:
|
||||
|
||||
```ini
|
||||
aws_region_name = your_region_like_eu-west-2
|
||||
aws_access_key_id = YOUR_ACCESS_KEY_ID
|
||||
aws_secret_access_key = YOUR_VERY_SECRET_ACCESS_KEY
|
||||
large_files_bucket_name = my_large_files
|
||||
endpoint_url = this is optional, for backblaze, use this: https://s3.us-west-002.backblazeb2.com
|
||||
```
|
||||
|
||||
> Just like in [example secrets](example_secrets.ini).
|
||||
|
||||
#### Print the expected options
|
||||
|
||||
```sh
|
||||
python3 -m large_file --help
|
||||
```
|
||||
|
||||
#### Upload some files
|
||||
|
||||
```sh
|
||||
python3 -m large_file --backend s3 --secrets secrets.ini --push my_first_file.json folder/my_second_file my_folder
|
||||
```
|
||||
|
||||
> Only the filename is used as the S3 name, the rest of the path is ignored.
|
||||
|
||||
#### Download some files to the local cache
|
||||
|
||||
This can be useful when building a Docker image for example. This way, the files can already reside inside the container and need not be downloaded later.
|
||||
|
||||
```sh
|
||||
python3 -m large_file --backend s3 -secrets ~/.aws/credentials --cache my_first_file.json:3 my_second_file my_folder:0
|
||||
```
|
||||
|
||||
> Versions may be specified by using `:`-s.
|
||||
|
||||
#### Delete remote files
|
||||
|
||||
```sh
|
||||
python3 -m large_file --backend s3 --secrets ~/.aws/credentials --delete my_first_file.json
|
||||
```
|
||||
|
|
@ -319,12 +319,14 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" checked>
|
||||
|
||||
|
||||
|
||||
|
|
@ -355,6 +357,98 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../configure-service/" class="md-nav__link">
|
||||
How to configure GreatAI
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../use-service/" class="md-nav__link">
|
||||
How to use a GreatAI service
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../handle-training-data/" class="md-nav__link">
|
||||
How to handle training data
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../large_file/" class="md-nav__link">
|
||||
How to use LargeFile
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../call_remote.md" class="md-nav__link">
|
||||
None
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
Scraping
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
|
@ -396,7 +490,7 @@
|
|||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
GreatAI reference
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
|
@ -458,6 +552,115 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5" type="checkbox" id="__nav_5" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_5">
|
||||
Examples
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Examples" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_5">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Examples
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5_1" type="checkbox" id="__nav_5_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_5_1">
|
||||
Explainable Naive Bayes
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Explainable Naive Bayes" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_5_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Explainable Naive Bayes
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../examples/simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
|
|
@ -1585,6 +1788,113 @@ async with httpx.AsyncClient(
|
|||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div><div class="jp-Cell jp-CodeCell jp-Notebook-cell ">
|
||||
<div class="jp-Cell jp-CodeCell jp-Notebook-cell ">
|
||||
<div class="jp-Cell-inputWrapper">
|
||||
<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
|
||||
</div>
|
||||
<div class="jp-InputArea jp-Cell-inputArea">
|
||||
<div class="jp-InputPrompt jp-InputArea-prompt">In [1]:</div><div class="jp-CodeMirrorEditor jp-Editor jp-InputArea-editor" data-type="inline">
|
||||
|
||||
|
||||
<div class="CodeMirror cm-s-jupyter">
|
||||
<div class="zeroclipboard-container">
|
||||
<clipboard-copy for="cell-2">
|
||||
<div>
|
||||
<span class="notice" hidden>Copied!</span>
|
||||
<svg aria-hidden="true" width="20" height="20" viewBox="0 0 16 16" version="1.1" data-view-component="true" class="clipboard-copy-icon">
|
||||
<path fill="currentColor" fill-rule="evenodd" d="M0 6.75C0 5.784.784 5 1.75 5h1.5a.75.75 0 010 1.5h-1.5a.25.25 0 00-.25.25v7.5c0 .138.112.25.25.25h7.5a.25.25 0 00.25-.25v-1.5a.75.75 0 011.5 0v1.5A1.75 1.75 0 019.25 16h-7.5A1.75 1.75 0 010 14.25v-7.5z"></path>
|
||||
<path fill="currentColor" fill-rule="evenodd" d="M5 1.75C5 .784 5.784 0 6.75 0h7.5C15.216 0 16 .784 16 1.75v7.5A1.75 1.75 0 0114.25 11h-7.5A1.75 1.75 0 015 9.25v-7.5zm1.75-.25a.25.25 0 00-.25.25v7.5c0 .138.112.25.25.25h7.5a.25.25 0 00.25-.25v-7.5a.25.25 0 00-.25-.25h-7.5z"></path>
|
||||
</svg>
|
||||
</div>
|
||||
</clipboard-copy>
|
||||
</div>
|
||||
<div class="highlight-ipynb hl-python"><pre><span></span><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">configure</span><span class="p">,</span> <span class="n">RouteConfig</span>
|
||||
<span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span>
|
||||
|
||||
|
||||
<span class="nd">@GreatAI</span><span class="o">.</span><span class="n">create</span>
|
||||
<span class="k">def</span> <span class="nf">greeter</span><span class="p">(</span><span class="n">your_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="sa">f</span><span class="s2">"Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s2">"</span>
|
||||
|
||||
|
||||
<span class="n">greeter</span><span class="p">(</span><span class="s2">"Bob"</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
<div id="cell-2" class="clipboard-copy-txt">from great_ai import configure, RouteConfig
|
||||
from great_ai import GreatAI
|
||||
|
||||
|
||||
@GreatAI.create
|
||||
def greeter(your_name: str) -> str:
|
||||
return f"Hi {your_name}"
|
||||
|
||||
|
||||
greeter("Bob")</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="jp-Cell-outputWrapper">
|
||||
<div class="jp-Collapser jp-OutputCollapser jp-Cell-outputCollapser">
|
||||
</div>
|
||||
|
||||
|
||||
<div class="jp-OutputArea jp-Cell-outputArea">
|
||||
|
||||
<div class="jp-OutputArea-child">
|
||||
|
||||
|
||||
<div class="jp-OutputPrompt jp-OutputArea-prompt"></div>
|
||||
|
||||
|
||||
<div class="jp-RenderedText jp-OutputArea-output" data-mime-type="application/vnd.jupyter.stderr">
|
||||
<pre><span style="color: rgb(255,255,0)">2022-07-11 19:00:27 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️</span>
|
||||
<span style="color: rgb(255,255,0)">2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver</span>
|
||||
<span style="color: rgb(255,255,0)">2022-07-11 19:00:27 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production</span>
|
||||
<span style="color: rgb(255,255,0)">2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal</span>
|
||||
<span style="color: rgb(0,175,255)">2022-07-11 19:00:27 | INFO | GreatAI (v0.1.3): configured ✅</span>
|
||||
<span style="color: rgb(0,175,255)">2022-07-11 19:00:27 | INFO | 🔩 tracing_database: ParallelTinyDbDriver</span>
|
||||
<span style="color: rgb(0,175,255)">2022-07-11 19:00:27 | INFO | 🔩 large_file_implementation: LargeFileLocal</span>
|
||||
<span style="color: rgb(0,175,255)">2022-07-11 19:00:27 | INFO | 🔩 is_production: False</span>
|
||||
<span style="color: rgb(0,175,255)">2022-07-11 19:00:27 | INFO | 🔩 should_log_exception_stack: True</span>
|
||||
<span style="color: rgb(0,175,255)">2022-07-11 19:00:27 | INFO | 🔩 prediction_cache_size: 512</span>
|
||||
<span style="color: rgb(0,175,255)">2022-07-11 19:00:27 | INFO | 🔩 dashboard_table_size: 50</span>
|
||||
<span style="color: rgb(255,255,0)">2022-07-11 19:00:27 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.</span>
|
||||
<span style="color: rgb(255,255,0)">2022-07-11 19:00:27 | WARNING | > Find out more at https://se-ml.github.io/practices</span>
|
||||
</pre>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="jp-OutputArea-child">
|
||||
|
||||
|
||||
<div class="jp-OutputPrompt jp-OutputArea-prompt">Out[1]:</div>
|
||||
|
||||
|
||||
|
||||
|
||||
<div class="jp-RenderedText jp-OutputArea-output jp-OutputArea-executeResult" data-mime-type="text/plain">
|
||||
<pre>Trace[str]({'created': '2022-07-11T17:00:27.064568',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.0896,
|
||||
'output': 'Hi Bob',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': '8ff5b268-2613-4e85-96ae-f248666a051f'})</pre>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -1596,7 +1906,7 @@ async with httpx.AsyncClient(
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 11, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -1613,6 +1923,42 @@ async with httpx.AsyncClient(
|
|||
|
||||
<footer class="md-footer">
|
||||
|
||||
|
||||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
<a href="../large_file/" class="md-footer__link md-footer__link--prev" aria-label="Previous: How to use LargeFile" rel="prev">
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</div>
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Previous
|
||||
</span>
|
||||
How to use LargeFile
|
||||
</div>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="../../reference/" class="md-footer__link md-footer__link--next" aria-label="Next: GreatAI reference" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
GreatAI reference
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
</nav>
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
|
|
|
|||
|
|
@ -16,13 +16,88 @@
|
|||
") as client:\n",
|
||||
" pass"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | GreatAI (v0.1.3): configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 tracing_database: ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 large_file_implementation: LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 dashboard_table_size: 50\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | > Find out more at https://se-ml.github.io/practices\u001b[0m\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Trace[str]({'created': '2022-07-11T17:00:27.064568',\n",
|
||||
" 'exception': None,\n",
|
||||
" 'feedback': None,\n",
|
||||
" 'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},\n",
|
||||
" 'models': [],\n",
|
||||
" 'original_execution_time_ms': 0.0896,\n",
|
||||
" 'output': 'Hi Bob',\n",
|
||||
" 'tags': ['greeter', 'online', 'development'],\n",
|
||||
" 'trace_id': '8ff5b268-2613-4e85-96ae-f248666a051f'})"
|
||||
]
|
||||
},
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from great_ai import configure, RouteConfig\n",
|
||||
"from great_ai import GreatAI\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@GreatAI.create\n",
|
||||
"def greeter(your_name: str) -> str:\n",
|
||||
" return f\"Hi {your_name}\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"greeter(\"Bob\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.10.4 ('.env': venv)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.4"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "02dd6d3afbfa9fbbe1037d64ad9014965528a1ccad21929d6e72f466389a68ad"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
|
|
|
|||
|
|
@ -1,586 +0,0 @@
|
|||
|
||||
<!doctype html>
|
||||
<html lang="en" class="no-js">
|
||||
<head>
|
||||
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
|
||||
<meta name="description" content="GreatAI helps you easily transform your prototype AI code into production-ready software.">
|
||||
|
||||
|
||||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
<title>Store models - GreatAI documentation</title>
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
|
||||
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../assets/_mkdocstrings.css">
|
||||
|
||||
<script>__md_scope=new URL("../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="light-blue" data-md-color-accent="">
|
||||
|
||||
|
||||
|
||||
<script>var palette=__md_get("__palette");if(palette&&"object"==typeof palette.color)for(var key of Object.keys(palette.color))document.body.setAttribute("data-md-color-"+key,palette.color[key])</script>
|
||||
|
||||
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
|
||||
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
|
||||
<label class="md-overlay" for="__drawer"></label>
|
||||
<div data-md-component="skip">
|
||||
|
||||
</div>
|
||||
<div data-md-component="announce">
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
<header class="md-header" data-md-component="header">
|
||||
<nav class="md-header__inner md-grid" aria-label="Header">
|
||||
<a href="../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
||||
</a>
|
||||
<label class="md-header__button md-icon" for="__drawer">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
|
||||
</label>
|
||||
<div class="md-header__title" data-md-component="header-title">
|
||||
<div class="md-header__ellipsis">
|
||||
<div class="md-header__topic">
|
||||
<span class="md-ellipsis">
|
||||
GreatAI documentation
|
||||
</span>
|
||||
</div>
|
||||
<div class="md-header__topic" data-md-component="header-topic">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Store models
|
||||
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<form class="md-header__option" data-md-component="palette">
|
||||
|
||||
|
||||
|
||||
<input class="md-option" data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme="default" data-md-color-primary="light-blue" data-md-color-accent="" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
|
||||
|
||||
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_2" hidden>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 2a7 7 0 0 0-7 7c0 2.38 1.19 4.47 3 5.74V17a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1v-2.26c1.81-1.27 3-3.36 3-5.74a7 7 0 0 0-7-7M9 21a1 1 0 0 0 1 1h4a1 1 0 0 0 1-1v-1H9v1Z"/></svg>
|
||||
</label>
|
||||
|
||||
|
||||
|
||||
|
||||
<input class="md-option" data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme="slate" data-md-color-primary="light-blue" data-md-color-accent="" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_2">
|
||||
|
||||
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 2a7 7 0 0 1 7 7c0 2.38-1.19 4.47-3 5.74V17a1 1 0 0 1-1 1H9a1 1 0 0 1-1-1v-2.26C6.19 13.47 5 11.38 5 9a7 7 0 0 1 7-7M9 21v-1h6v1a1 1 0 0 1-1 1h-4a1 1 0 0 1-1-1m3-17a5 5 0 0 0-5 5c0 2.05 1.23 3.81 3 4.58V16h4v-2.42c1.77-.77 3-2.53 3-4.58a5 5 0 0 0-5-5Z"/></svg>
|
||||
</label>
|
||||
|
||||
|
||||
</form>
|
||||
|
||||
|
||||
|
||||
<label class="md-header__button md-icon" for="__search">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
|
||||
</label>
|
||||
<div class="md-search" data-md-component="search" role="dialog">
|
||||
<label class="md-search__overlay" for="__search"></label>
|
||||
<div class="md-search__inner" role="search">
|
||||
<form class="md-search__form" name="search">
|
||||
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
|
||||
<label class="md-search__icon md-icon" for="__search">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</label>
|
||||
<nav class="md-search__options" aria-label="Search">
|
||||
|
||||
<button type="reset" class="md-search__icon md-icon" aria-label="Clear" tabindex="-1">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41Z"/></svg>
|
||||
</button>
|
||||
</nav>
|
||||
|
||||
</form>
|
||||
<div class="md-search__output">
|
||||
<div class="md-search__scrollwrap" data-md-scrollfix>
|
||||
<div class="md-search-result" data-md-component="search-result">
|
||||
<div class="md-search-result__meta">
|
||||
Initializing search
|
||||
</div>
|
||||
<ol class="md-search-result__list"></ol>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="md-header__source">
|
||||
<a href="https://github.com/schmelczer/great-ai" title="Go to repository" class="md-source" data-md-component="source">
|
||||
<div class="md-source__icon md-icon">
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
|
||||
</div>
|
||||
<div class="md-source__repository">
|
||||
schmelczer/great-ai
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
</nav>
|
||||
|
||||
</header>
|
||||
|
||||
<div class="md-container" data-md-component="container">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<main class="md-main" data-md-component="main">
|
||||
<div class="md-main__inner md-grid">
|
||||
|
||||
|
||||
|
||||
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
|
||||
<div class="md-sidebar__scrollwrap">
|
||||
<div class="md-sidebar__inner">
|
||||
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
|
||||
<label class="md-nav__title" for="__drawer">
|
||||
<a href="../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
||||
</a>
|
||||
GreatAI documentation
|
||||
</label>
|
||||
|
||||
<div class="md-nav__source">
|
||||
<a href="https://github.com/schmelczer/great-ai" title="Go to repository" class="md-source" data-md-component="source">
|
||||
<div class="md-source__icon md-icon">
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
|
||||
</div>
|
||||
<div class="md-source__repository">
|
||||
schmelczer/great-ai
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../.." class="md-nav__link">
|
||||
Overview
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/" class="md-nav__link">
|
||||
Tutorial overview
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../create-service/" class="md-nav__link">
|
||||
How to create a GreatAI service
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_4" type="checkbox" id="__nav_4" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_4">
|
||||
Reference
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Reference" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_4">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Reference
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../reference/utilities/" class="md-nav__link">
|
||||
Utilities
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../reference/large-file/" class="md-nav__link">
|
||||
LargeFile
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../reference/views/" class="md-nav__link">
|
||||
View models
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
|
||||
<div class="md-sidebar__scrollwrap">
|
||||
<div class="md-sidebar__inner">
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
|
||||
|
||||
|
||||
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="md-content" data-md-component="content">
|
||||
<article class="md-content__inner md-typeset">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/store-models.md" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<h1>Store models</h1>
|
||||
|
||||
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 10, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
</article>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</main>
|
||||
|
||||
<footer class="md-footer">
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
|
||||
<div class="md-copyright__highlight">
|
||||
GNU General Public License v3
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
<div class="md-social">
|
||||
|
||||
|
||||
|
||||
<a href="https://schmelczer.dev" target="_blank" rel="noopener" title="about the author" class="md-social__link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="m414.8 40.79-128 448.01c-4.9 17-22.6 26.8-39.6 22-17-4.9-26.8-22.6-22-39.6l128-447.99c4.9-16.994 22.6-26.834 39.6-21.978 17 4.855 26.8 22.568 22 39.558zm103.8 80.61 112 112c12.5 12.5 12.5 32.7 0 45.2l-112 112c-12.5 12.5-32.7 12.5-45.2 0s-12.5-32.7 0-45.2l89.3-89.4-89.3-89.4c-12.5-12.5-12.5-32.7 0-45.2s32.7-12.5 45.2 0zm-352 45.2L77.25 256l89.35 89.4c12.5 12.5 12.5 32.7 0 45.2s-32.7 12.5-45.2 0L9.372 278.6c-12.496-12.5-12.496-32.7 0-45.2l112.028-112c12.5-12.5 32.7-12.5 45.2 0s12.5 32.7 0 45.2z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai" target="_blank" rel="noopener" title="great-ai on Github" class="md-social__link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 496 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="https://hub.docker.com/repository/docker/schmelczera/great-ai" target="_blank" rel="noopener" title="great-ai on DockerHub" class="md-social__link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M349.9 236.3h-66.1v-59.4h66.1v59.4zm0-204.3h-66.1v60.7h66.1V32zm78.2 144.8H362v59.4h66.1v-59.4zm-156.3-72.1h-66.1v60.1h66.1v-60.1zm78.1 0h-66.1v60.1h66.1v-60.1zm276.8 100c-14.4-9.7-47.6-13.2-73.1-8.4-3.3-24-16.7-44.9-41.1-63.7l-14-9.3-9.3 14c-18.4 27.8-23.4 73.6-3.7 103.8-8.7 4.7-25.8 11.1-48.4 10.7H2.4c-8.7 50.8 5.8 116.8 44 162.1 37.1 43.9 92.7 66.2 165.4 66.2 157.4 0 273.9-72.5 328.4-204.2 21.4.4 67.6.1 91.3-45.2 1.5-2.5 6.6-13.2 8.5-17.1l-13.3-8.9zm-511.1-27.9h-66v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm-78.1-72.1h-66.1v60.1h66.1v-60.1z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="https://pypi.org/project/great-ai" target="_blank" rel="noopener" title="great-ai on PyPI" class="md-social__link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg>
|
||||
</a>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</footer>
|
||||
|
||||
</div>
|
||||
<div class="md-dialog" data-md-component="dialog">
|
||||
<div class="md-dialog__inner md-typeset"></div>
|
||||
</div>
|
||||
<script id="__config" type="application/json">{"base": "../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
|
||||
|
||||
<script src="../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
1098
how-to-guides/use-service/index.html
Normal file
1098
how-to-guides/use-service/index.html
Normal file
File diff suppressed because it is too large
Load diff
131
how-to-guides/use-service/use-service.md
Normal file
131
how-to-guides/use-service/use-service.md
Normal file
|
|
@ -0,0 +1,131 @@
|
|||
# How to use a GreatAI service
|
||||
|
||||
After [creating a GreatAI service](/how-to-guides/create-service) by wrapping your prediction function, and optionally [configuring it](/how-to-guides/configure-service), it's time to do some prediction.
|
||||
|
||||
Let's take the following example:
|
||||
|
||||
```python title="greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
||||
@GreatAI.create
|
||||
def greeter(your_name: str) -> str:
|
||||
return f'Hi {your_name}'
|
||||
```
|
||||
|
||||
## One-off prediction
|
||||
|
||||
Even though `greeter` is now an instance of [GreatAI][great_ai.GreatAI], you can continue using it as a regular function.
|
||||
|
||||
```python
|
||||
>>> greeter('Bob')
|
||||
Trace[str]({'created': '2022-07-11T14:31:46.183764',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.0381,
|
||||
'output': 'Hi Bob',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': '7c284fd7-7f0d-4464-b5f8-3ef126df34af'})
|
||||
```
|
||||
|
||||
As you can see, the original return value is wrapped in a [Trace][great_ai.Trace] object (which is also persisted in your database of choice). You can access the original value under the `output` property.
|
||||
|
||||
## Online prediction
|
||||
|
||||
Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
|
||||
|
||||
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To this end, `uvicorn` is wrapped by the `great-ai` command-line utility which, among others, takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
|
||||
|
||||
### In development
|
||||
|
||||
```sh
|
||||
great-ai greeter.py
|
||||
```
|
||||
|
||||
!!! success
|
||||
Your model is accessible at [localhost:6060](http:/127.0.0.1:6060){ target=_blank }.
|
||||
|
||||
Some configuration options are also supported.
|
||||
|
||||
```sh
|
||||
great-ai greeter.py --port 8000 --host 127.0.0.1 --timeout_keep_alive 10
|
||||
```
|
||||
> For more options (but no Notebook support, use [uvicorn](https://www.uvicorn.org/){ target=_blank })
|
||||
|
||||
### In production
|
||||
|
||||
There are three main approaches for deploying a GreatAI service.
|
||||
|
||||
#### Manual deployment
|
||||
|
||||
The app is run in *production-mode* if the value of the `ENVIRONMENT` environment variable is set to `production`.
|
||||
|
||||
```sh
|
||||
ENVIRONMENT=production great-ai greeter.py
|
||||
```
|
||||
|
||||
Simply run `ENVIRONMENT=production great-ai deploy.ipynb` in the command-line of a production machine.
|
||||
> This is the crudest approach, however, it might be fitting for some contexts.
|
||||
|
||||
#### Containerised deployment
|
||||
|
||||
Run the notebook directly in a container or create a service for it using your favourite container orchestrator.
|
||||
|
||||
```sh
|
||||
docker run -p 6060:6060 --volume `pwd`:/app --rm \
|
||||
schmelczera/great-ai deploy.ipynb
|
||||
```
|
||||
> You can replace ``pwd`` with the path to your code's folder.
|
||||
|
||||
#### Use a Platform-as-a-Service
|
||||
|
||||
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank } or [MLEM](https://mlem.ai/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
|
||||
|
||||
To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.
|
||||
|
||||
```Dockerfile
|
||||
FROM schmelczera/great-ai:latest
|
||||
|
||||
# Remove this block if you don't have a requirements.txt
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --no-cache-dir --requirement requirements.txt
|
||||
|
||||
# If you store your models in S3 or GridFS, it may be a
|
||||
# good idea to cache them in the image so that you don't
|
||||
# have to download it each time a container starts
|
||||
RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor
|
||||
|
||||
# Add you application code to the image
|
||||
COPY . .
|
||||
|
||||
# The default ENTRYPOINT is great-ai, specify it's argument using CMD
|
||||
CMD ["deploy.ipynb"]
|
||||
|
||||
```
|
||||
|
||||
## Batch prediction
|
||||
|
||||
Processing larger amounts of data on a single machine is made easy by the [GreatAI][great_ai.GreatAI]'s [process_batch][great_ai.GreatAI.process_batch] method. This relies on multiprocessing ([parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map]) to take full advantage of all available CPU-cores.
|
||||
|
||||
```python
|
||||
>>> greeter.process_batch(['Alice', 'Bob'])
|
||||
[Trace[str]({'created': '2022-07-11T14:36:37.119183',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 5, 'arg:your_name:value': 'Alice'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.1251,
|
||||
'output': 'Hi Alice',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': '90ffa15f-e839-41c4-8e7a-3211168bc138'}),
|
||||
Trace[str]({'created': '2022-07-11T14:36:37.166659',
|
||||
'exception': None,
|
||||
'feedback': None,
|
||||
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
|
||||
'models': [],
|
||||
'original_execution_time_ms': 0.0571,
|
||||
'output': 'Hi Bob',
|
||||
'tags': ['greeter', 'online', 'development'],
|
||||
'trace_id': 'f48e94c7-0815-48b3-a864-41349d3dae84'})]
|
||||
```
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
# How to use a GreatAI service
|
||||
|
||||
After [creating a GreatAI service](/how-to-guides/cerate-service) by wrapping your prediction function, it's time to do some prediction.
|
||||
|
||||
Let's use the following example:
|
||||
|
||||
```python "type_safe_greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
||||
@GreatAI.create
|
||||
def type_safe_greeter(your_name: str) -> str:
|
||||
return f'Hi {your_name}'
|
||||
```
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue