andras/great-ai
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 4 Packages Wiki Activity Actions

Deployed 9999b7e with MkDocs version: 1.3.0

Browse source
This commit is contained in:
2022-07-15 18:04:53 +00:00
parent 087e049514
commit 7d09422380
37 changed files with 116 additions and 108 deletions
Download patch file Download diff file Expand all files Collapse all files

4
how-to-guides/call-remote/call-remote.md
View file

@ -56,7 +56,7 @@ As you can see, everything worked as expected. There is one way to improve it th
Let's send multiple requests at the same time to speed up the overall execution time. To do this, we will use the [call_remote_great_ai_async][great_ai.call_remote_great_ai_async] function.
??? note "Why is this possible?"
Note, that in `server.py`, the inference function is declared `async`. This means that multiple "copies" of it can run at the same time in the same thread. Since, there is no CPU-bottleneck, the server has a quite large throughpout (requests responded to per second), but its latency will stay around 2 seconds due to the async `sleep` command.
Note, that in `server.py`, the inference function is declared `async`. This means that multiple "copies" of it can run at the same time in the same thread. Since, there is no CPU bottleneck, the server has a quite large throughput (requests responded to per second), but its latency will stay around 2 seconds due to the async `sleep` command.
If your great-ai server is not `async`, higher throughput can be achieved by running multiple instances of it, either manually, or by running it with multiple `uvicorn` workers like this: `ENVIRONMENT=production great-ai server.py --worker_count 4`
@ -88,4 +88,4 @@ asyncio.run(main())
![screenshot of result](/media/remote-async.png){ loading=lazy }
This also works, and in some use cases might be considerably quicker.
This also works, and in some use-cases might be considerably quicker.

12
how-to-guides/call-remote/index.html
View file

@ -438,7 +438,7 @@
<li class="md-nav__item">
<a href="../large_file/" class="md-nav__link">
How to use LargeFile-s
How to use LargeFiles
</a>
</li>
@ -919,7 +919,7 @@
<p>Let's send multiple requests at the same time to speed up the overall execution time. To do this, we will use the <a class="autorefs autorefs-internal" href="../../reference/#great_ai.call_remote_great_ai_async">call_remote_great_ai_async</a> function.</p>
<details class="note">
<summary>Why is this possible?</summary>
<p>Note, that in <code>server.py</code>, the inference function is declared <code>async</code>. This means that multiple "copies" of it can run at the same time in the same thread. Since, there is no CPU-bottleneck, the server has a quite large throughpout (requests responded to per second), but its latency will stay around 2 seconds due to the async <code>sleep</code> command.</p>
<p>Note, that in <code>server.py</code>, the inference function is declared <code>async</code>. This means that multiple "copies" of it can run at the same time in the same thread. Since, there is no CPU bottleneck, the server has a quite large throughput (requests responded to per second), but its latency will stay around 2 seconds due to the async <code>sleep</code> command.</p>
<p>If your great-ai server is not <code>async</code>, higher throughput can be achieved by running multiple instances of it, either manually, or by running it with multiple <code>uvicorn</code> workers like this: <code>ENVIRONMENT=production great-ai server.py --worker_count 4</code></p>
</details>
<h3 id="async-client">Async client<a class="headerlink" href="#async-client" title="Permanent link">#</a></h3>
@ -947,14 +947,14 @@
<p>Replace <code>client.py</code> with this async client. Note that even though async support is significantly more streamlined in recent Python versions, it still requires a bit more boilerplate than its synchronous counterpart.</p>
</blockquote>
<p><img alt="screenshot of result" loading="lazy" src="/media/remote-async.png" /></p>
<p>This also works, and in some use cases might be considerably quicker.</p>
<p>This also works, and in some use-cases might be considerably quicker.</p>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 13, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
</small>
@ -975,7 +975,7 @@
<nav class="md-footer__inner md-grid" aria-label="Footer" >
<a href="../large_file/" class="md-footer__link md-footer__link--prev" aria-label="Previous: How to use LargeFile-s" rel="prev">
<a href="../large_file/" class="md-footer__link md-footer__link--prev" aria-label="Previous: How to use LargeFiles" rel="prev">
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</div>
@ -984,7 +984,7 @@
<span class="md-footer__direction">
Previous
</span>
How to use LargeFile-s
How to use LargeFiles
</div>
</div>
</a>

8
how-to-guides/configure-service/configure-service.md
View file

@ -1,6 +1,6 @@
# How to configure GreatAI
GreatAI aims to provide reasonable defaults wherever possible. The current configuration is always prominently displayed (and updated) on the dashboard and in the command-line startup banner.
GreatAI aims to provide reasonable defaults wherever possible. The current configuration is always prominently displayed (and updated) on the dashboard and in the command-line start-up banner.
## Using [great_ai.configure][]
@ -37,7 +37,7 @@ The only aspect that cannot be automated is choosing the backing storage for the
Right now, you have 3 options for storing the models and large datasets: [LargeFileLocal][great_ai.large_file.LargeFileLocal], [LargeFileMongo][great_ai.large_file.LargeFileMongo], and [LargeFileS3][great_ai.large_file.LargeFileS3].
Without explicit configuration, [LargeFileLocal][great_ai.large_file.LargeFileLocal] is selected by default. This one still version-controls your files but it only stores them in a local path.
Without explicit configuration, [LargeFileLocal][great_ai.large_file.LargeFileLocal] is selected by default. This one still version-controls your files but it only stores them in a local path (which of course can be a remote volume attached by [NFS](https://en.wikipedia.org/wiki/Network_File_System){ target=_blank }, [HDFS](https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html){ target=_blank }, etc.).
!!! important
If your working directory contains a `mongo.ini` or `s3.ini` file, an attempt is made to auto-configure [LargeFileMongo][great_ai.large_file.LargeFileMongo] or [LargeFileS3][great_ai.large_file.LargeFileS3] respectively.
@ -63,7 +63,7 @@ model = [4, 3]
save_model(model, 'my-model')
```
1. This line isn't strictly necceseary because if `s3.ini` (or `mongo.ini`) is available in the current working directory, they are automatically used to configure their respective LargeFile implementations/databases.
1. This line isn't strictly necessary because if `s3.ini` (or `mongo.ini`) is available in the current working directory, they are automatically used to configure their respective LargeFile implementations/databases.
??? note "Departing from AWS"
With the `aws_endpoint_url` argument, it is possible to use any other S3-compatible service such as [Backblaze](https://www.backblaze.com/){ target=_blank }. In that case, it would be `aws_endpoint_url=https://s3.us-west-002.backblazeb2.com`.
@ -99,4 +99,4 @@ By default, a thread-safe version of [TinyDB](https://tinydb.readthedocs.io/en/l
### MongoDB
At the moment, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory, or explicitly call [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].
At the moment, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory, or explicitly call either [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].

12
how-to-guides/configure-service/index.html
View file

@ -526,7 +526,7 @@
<li class="md-nav__item">
<a href="../large_file/" class="md-nav__link">
How to use LargeFile-s
How to use LargeFiles
</a>
</li>
@ -884,7 +884,7 @@
<h1 id="how-to-configure-greatai">How to configure GreatAI<a class="headerlink" href="#how-to-configure-greatai" title="Permanent link">#</a></h1>
<p>GreatAI aims to provide reasonable defaults wherever possible. The current configuration is always prominently displayed (and updated) on the dashboard and in the command-line startup banner.</p>
<p>GreatAI aims to provide reasonable defaults wherever possible. The current configuration is always prominently displayed (and updated) on the dashboard and in the command-line start-up banner.</p>
<h2 id="using-great_aiconfigure">Using <a class="autorefs autorefs-internal" href="../../reference/#great_ai.configure">great_ai.configure</a><a class="headerlink" href="#using-great_aiconfigure" title="Permanent link">#</a></h2>
<p>You can override any of the default settings by calling <a class="autorefs autorefs-internal" href="../../reference/#great_ai.configure">great_ai.configure</a>. If you don't call <code>configure</code>, the default settings are applied on the first call to most <code>great-ai</code> functions.</p>
<div class="admonition warning">
@ -915,7 +915,7 @@
<h2 id="using-remote-storage">Using remote storage<a class="headerlink" href="#using-remote-storage" title="Permanent link">#</a></h2>
<p>The only aspect that cannot be automated is choosing the backing storage for the database and file storage.</p>
<p>Right now, you have 3 options for storing the models and large datasets: <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileLocal">LargeFileLocal</a>, <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileMongo">LargeFileMongo</a>, and <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileS3">LargeFileS3</a>.</p>
<p>Without explicit configuration, <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileLocal">LargeFileLocal</a> is selected by default. This one still version-controls your files but it only stores them in a local path.</p>
<p>Without explicit configuration, <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileLocal">LargeFileLocal</a> is selected by default. This one still version-controls your files but it only stores them in a local path (which of course can be a remote volume attached by <a href="https://en.wikipedia.org/wiki/Network_File_System" target="_blank">NFS</a>, <a href="https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html" target="_blank">HDFS</a>, etc.).</p>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>If your working directory contains a <code>mongo.ini</code> or <code>s3.ini</code> file, an attempt is made to auto-configure <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileMongo">LargeFileMongo</a> or <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileS3">LargeFileS3</a> respectively.</p>
@ -936,7 +936,7 @@
<a id="__codelineno-2-7" name="__codelineno-2-7" href="#__codelineno-2-7"></a><span class="n">save_model</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="s1">&#39;my-model&#39;</span><span class="p">)</span>
</code></pre></div>
<ol>
<li>This line isn't strictly necceseary because if <code>s3.ini</code> (or <code>mongo.ini</code>) is available in the current working directory, they are automatically used to configure their respective LargeFile implementations/databases.</li>
<li>This line isn't strictly necessary because if <code>s3.ini</code> (or <code>mongo.ini</code>) is available in the current working directory, they are automatically used to configure their respective LargeFile implementations/databases.</li>
</ol>
<details class="note">
<summary>Departing from AWS</summary>
@ -965,14 +965,14 @@
<h2 id="using-a-database">Using a database<a class="headerlink" href="#using-a-database" title="Permanent link">#</a></h2>
<p>By default, a thread-safe version of <a href="https://tinydb.readthedocs.io/en/latest/" target="_blank">TinyDB</a> is utilised for saving the prediction traces into a local file. Unfortunately, for most production needs, this method is not suitable.</p>
<h3 id="mongodb">MongoDB<a class="headerlink" href="#mongodb" title="Permanent link">#</a></h3>
<p>At the moment, only MongoDB is supported as a production-ready <code>TracingDatabase</code>. In order to use it, you have to either place a file named <code>mongo.ini</code> in your working directory, or explicitly call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.MongoDbDriver">MongoDbDriver.configure_credentials_from_file</a> or <a class="autorefs autorefs-internal" href="../../reference/#great_ai.persistence.mongodb_driver.MongoDbDriver.configure_credentials">MongoDbDriver.configure_credentials</a>.</p>
<p>At the moment, only MongoDB is supported as a production-ready <code>TracingDatabase</code>. In order to use it, you have to either place a file named <code>mongo.ini</code> in your working directory, or explicitly call either <a class="autorefs autorefs-internal" href="../../reference/#great_ai.MongoDbDriver">MongoDbDriver.configure_credentials_from_file</a> or <a class="autorefs autorefs-internal" href="../../reference/#great_ai.persistence.mongodb_driver.MongoDbDriver.configure_credentials">MongoDbDriver.configure_credentials</a>.</p>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 12, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
</small>

12
how-to-guides/create-service/create-service.md
View file

@ -20,7 +20,7 @@ def greeter(your_name):
```
??? info "Why not simply use `@GreatAI?`"
The purpose of the [@GreatAI.create][great_ai.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
The purpose of [@GreatAI.create][great_ai.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
## With types
@ -46,13 +46,13 @@ from asyncio import sleep
@GreatAI.create
async def async_greeter(your_name: str) -> str:
await sleep(2) # simulate IO-heavy operation
await sleep(2) # simulate IO-bound operation
return f'Hi {your_name}!'
```
## With decorators
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.GreatAI.create] always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.GreatAI.create] must come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
### Using `@use_model`
@ -72,7 +72,7 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
1. By default, the parameter named `model` will be replaced by the loaded model. This behaviour can be customised by setting the `model_kwarg_name`. This way, even multiple models can be injected into a single function.
!!! important
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.GreatAI.create]. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.GreatAI.create]. Note, that decorators are applied starting from the bottom-most one. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
### Using `@parameter`
@ -93,7 +93,7 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
```
!!! important
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.GreatAI.create]. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.GreatAI.create]. Note, that decorators are applied starting from the bottom-most one. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
## Complex example
@ -117,4 +117,4 @@ def add_number(positive_number: int, secret: int) -> int:
assert add_number(1).output == 5
```
1. Refer to [storing models](/how-to-guides/store-models) for specifying where to store your models.
1. Refer to [the configuration page](/how-to-guides/configure-service) for specifying where to store your models.

16
how-to-guides/create-service/index.html
View file

@ -520,7 +520,7 @@
<li class="md-nav__item">
<a href="../large_file/" class="md-nav__link">
How to use LargeFile-s
How to use LargeFiles
</a>
</li>
@ -886,7 +886,7 @@
</code></pre></div>
<details class="info">
<summary>Why not simply use <code>@GreatAI?</code></summary>
<p>The purpose of the <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for <code>__new__</code> is lacking in MyPy, thus, a static factory method is used instead.</p>
<p>The purpose of <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for <code>__new__</code> is lacking in MyPy, thus, a static factory method is used instead.</p>
</details>
<h2 id="with-types">With types<a class="headerlink" href="#with-types" title="Permanent link">#</a></h2>
<p>Even though it's not required by GreatAI, <a href="https://realpython.com/python-type-checking/" target="_blank">type annotating your codebase</a> can save you from lots of trivial mistakes, that's why it's highly advised. Simply add the expected types to your function's signature.</p>
@ -904,11 +904,11 @@
<a id="__codelineno-3-3" name="__codelineno-3-3" href="#__codelineno-3-3"></a>
<a id="__codelineno-3-4" name="__codelineno-3-4" href="#__codelineno-3-4"></a><span class="nd">@GreatAI</span><span class="o">.</span><span class="n">create</span>
<a id="__codelineno-3-5" name="__codelineno-3-5" href="#__codelineno-3-5"></a><span class="k">async</span> <span class="k">def</span> <span class="nf">async_greeter</span><span class="p">(</span><span class="n">your_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
<a id="__codelineno-3-6" name="__codelineno-3-6" href="#__codelineno-3-6"></a> <span class="k">await</span> <span class="n">sleep</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># simulate IO-heavy operation</span>
<a id="__codelineno-3-6" name="__codelineno-3-6" href="#__codelineno-3-6"></a> <span class="k">await</span> <span class="n">sleep</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># simulate IO-bound operation</span>
<a id="__codelineno-3-7" name="__codelineno-3-7" href="#__codelineno-3-7"></a> <span class="k">return</span> <span class="sa">f</span><span class="s1">&#39;Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s1">!&#39;</span>
</code></pre></div>
<h2 id="with-decorators">With decorators<a class="headerlink" href="#with-decorators" title="Permanent link">#</a></h2>
<p>GreatAI can decorate already decorated functions. The only restriction is that <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> always has to come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.</p>
<p>GreatAI can decorate already decorated functions. The only restriction is that <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> must come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.</p>
<h3 id="using-use_model">Using <code>@use_model</code><a class="headerlink" href="#using-use_model" title="Permanent link">#</a></h3>
<p>If you have previously saved a model with <a class="autorefs autorefs-internal" href="../../reference/#great_ai.save_model">save_model</a>, you can inject it into your function by calling <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a>.</p>
<div class="highlight"><span class="filename">greeter_with_model.py</span><pre><span></span><code><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span><span class="p">,</span> <span class="n">use_model</span>
@ -925,7 +925,7 @@
</ol>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> in other places of the codebase, it works equally well outside of GreatAI services. </p>
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Note, that decorators are applied starting from the bottom-most one. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> in other places of the codebase, it works equally well outside of GreatAI services. </p>
</div>
<h3 id="using-parameter">Using <code>@parameter</code><a class="headerlink" href="#using-parameter" title="Permanent link">#</a></h3>
<p>If you wish to turn off logging or specify custom validation for your parameters, you can use the <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> decorator.</p>
@ -944,7 +944,7 @@
</code></pre></div>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> in other places of the codebase, it works equally well outside of GreatAI services. </p>
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Note, that decorators are applied starting from the bottom-most one. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> in other places of the codebase, it works equally well outside of GreatAI services. </p>
</div>
<h2 id="complex-example">Complex example<a class="headerlink" href="#complex-example" title="Permanent link">#</a></h2>
<p>Refer to the following example summarising the options you have when instantiating a GreatAI service.</p>
@ -965,7 +965,7 @@
<a id="__codelineno-6-15" name="__codelineno-6-15" href="#__codelineno-6-15"></a><span class="k">assert</span> <span class="n">add_number</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">output</span> <span class="o">==</span> <span class="mi">5</span>
</code></pre></div>
<ol>
<li>Refer to <a href="/how-to-guides/store-models">storing models</a> for specifying where to store your models. </li>
<li>Refer to <a href="/how-to-guides/configure-service">the configuration page</a> for specifying where to store your models. </li>
</ol>
<hr>
@ -973,7 +973,7 @@
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 13, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
</small>

6
how-to-guides/handle-training-data/handle-training-data.md
View file

@ -7,9 +7,9 @@ In order to simplify your training data management, `great-ai` provide two compl
At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call [great_ai.add_ground_truth][]. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, [great_ai.query_ground_truth][] can be called to get a filtered view of the training data.
!!! important "Train-test-validation splits"
It is a best-practice to lock-away a test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.
It is a best-practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.
With [great_ai.add_ground_truth][], there is an option to tag the samples with `train`, `test`, and `validation` randomly, following a predefined distribution. This happens as soon as they're written in the database. Later on, these can be queried by providing the name of the appropriate tags.
With [great_ai.add_ground_truth][], there is an option to tag the samples with `train`, `test`, and `validation` randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.
The nice thing about this is that the 'input-expected output' pairs are stored as traces. Thus, they behave exactly like regular prediction traces.
@ -52,7 +52,7 @@ add_ground_truth(
1. Expected output. This can be also accessed through the `.output` property.
2. The input value is stored here.
3. Notice how `ground_truth` always included as a tag when using [great_ai.add_ground_truth][].
3. Notice how `ground_truth` is always included as a tag when using [great_ai.add_ground_truth][].
## Get feedback

14
how-to-guides/handle-training-data/index.html
View file

@ -493,7 +493,7 @@
<li class="md-nav__item">
<a href="../large_file/" class="md-nav__link">
How to use LargeFile-s
How to use LargeFiles
</a>
</li>
@ -823,8 +823,8 @@
<p>At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, <a class="autorefs autorefs-internal" href="../../reference/#great_ai.query_ground_truth">great_ai.query_ground_truth</a> can be called to get a filtered view of the training data.</p>
<div class="admonition important">
<p class="admonition-title">Train-test-validation splits</p>
<p>It is a best-practice to lock-away a test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.</p>
<p>With <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>, there is an option to tag the samples with <code>train</code>, <code>test</code>, and <code>validation</code> randomly, following a predefined distribution. This happens as soon as they're written in the database. Later on, these can be queried by providing the name of the appropriate tags.</p>
<p>It is a best-practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.</p>
<p>With <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>, there is an option to tag the samples with <code>train</code>, <code>test</code>, and <code>validation</code> randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.</p>
</div>
<p>The nice thing about this is that the 'input-expected output' pairs are stored as traces. Thus, they behave exactly like regular prediction traces.</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">add_ground_truth</span>
@ -864,7 +864,7 @@
<ol>
<li>Expected output. This can be also accessed through the <code>.output</code> property.</li>
<li>The input value is stored here.</li>
<li>Notice how <code>ground_truth</code> always included as a tag when using <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>. </li>
<li>Notice how <code>ground_truth</code> is always included as a tag when using <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>. </li>
</ol>
<h2 id="get-feedback">Get feedback<a class="headerlink" href="#get-feedback" title="Permanent link">#</a></h2>
<p>After the initial data gathering, end-to-end feedback can be also integrated into the dataset. </p>
@ -879,7 +879,7 @@
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 12, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
</small>
@ -916,13 +916,13 @@
<a href="../large_file/" class="md-footer__link md-footer__link--next" aria-label="Next: How to use LargeFile-s" rel="next">
<a href="../large_file/" class="md-footer__link md-footer__link--next" aria-label="Next: How to use LargeFiles" rel="next">
<div class="md-footer__title">
<div class="md-ellipsis">
<span class="md-footer__direction">
Next
</span>
How to use LargeFile-s
How to use LargeFiles
</div>
</div>
<div class="md-footer__button md-icon">

15
how-to-guides/install/index.html
View file

@ -486,7 +486,7 @@
<li class="md-nav__item">
<a href="../large_file/" class="md-nav__link">
How to use LargeFile-s
How to use LargeFiles
</a>
</li>
@ -813,12 +813,17 @@
<p>This will work on all major operating systems.</p>
<h2 id="command-line-tools">Command-line tools<a class="headerlink" href="#command-line-tools" title="Permanent link">#</a></h2>
<p>After installation, <code>great-ai</code> and <code>large-file</code> are available as commands. The former is required for deploying your application while the latter lets you manage models and datasets from your terminal.</p>
<details class="note">
<summary>Snakes &amp; kebabs</summary>
<p>The library is called <code>great-ai</code>, therefore, its command-line entrypoint is also called <code>great-ai</code>. However, Python module names cannot contain hyphens, that's why you have to <code>import great_ai</code> with an underscore. The <code>great-ai</code> CLI tool is also available as <code>python3 -m great_ai</code>.</p>
<p>To help with the confusion, a CLI executable called <code>great_ai</code> (and <code>large_file</code>) are also installed, thus, if you prefer, you can always refer to GreatAI using its underscored name variant (<code>great_ai</code>).</p>
</details>
<div class="admonition warning">
<p class="admonition-title">Windows</p>
<p>On Windows, you might encounter a similar <mark>warning</mark> from <code>pip</code>:</p>
<p>On Windows, you might encounter a similar warning from <code>pip</code>:</p>
<blockquote>
<p>WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users...\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.</p>
<p><code>WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users\...\Scripts' which is not on PATH.</code></p>
<p><code>Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.</code></p>
</blockquote>
<p>This means that <code>great-ai.exe</code> and <code>large-file.exe</code> are not in your <code>PATH</code>. Either add their containing directory ('C:\Users...\Scripts' in this case) to your <code>PATH</code> or use <code>python3 -m great_ai</code> and <code>python3 -m great_ai.large_file</code> instead of the exe-s.</p>
</div>
@ -832,7 +837,7 @@ Consider adding this directory to PATH or, if you prefer to suppress this warnin
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 13, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
</small>

12
how-to-guides/install/install.md
View file

@ -13,10 +13,16 @@ This will work on all major operating systems.
After installation, `great-ai` and `large-file` are available as commands. The former is required for deploying your application while the latter lets you manage models and datasets from your terminal.
??? note "Snakes & kebabs"
The library is called `great-ai`, therefore, its command-line entrypoint is also called `great-ai`. However, Python module names cannot contain hyphens, that's why you have to `import great_ai` with an underscore. The `great-ai` CLI tool is also available as `python3 -m great_ai`.
To help with the confusion, a CLI executable called `great_ai` (and `large_file`) are also installed, thus, if you prefer, you can always refer to GreatAI using its underscored name variant (`great_ai`).
!!! warning "Windows"
On Windows, you might encounter a similar ==warning== from `pip`:
> WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users\...\Scripts' which is not on PATH.
> Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
On Windows, you might encounter a similar warning from `pip`:
> `WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users\...\Scripts' which is not on PATH.`
> `Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.`
This means that `great-ai.exe` and `large-file.exe` are not in your `PATH`. Either add their containing directory ('C:\Users\...\Scripts' in this case) to your `PATH` or use `python3 -m great_ai` and `python3 -m great_ai.large_file` instead of the exe-s.

26
how-to-guides/large_file/index.html
View file

@ -17,7 +17,7 @@
<title>How to use LargeFile-s - GreatAI documentation</title>
<title>How to use LargeFiles - GreatAI documentation</title>
@ -83,7 +83,7 @@
<div data-md-component="skip">
<a href="#how-to-use-largefile-s" class="md-skip">
<a href="#how-to-use-largefiles" class="md-skip">
Skip to content
</a>
@ -116,7 +116,7 @@
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
How to use LargeFile-s
How to use LargeFiles
</span>
</div>
@ -447,12 +447,12 @@
<label class="md-nav__link md-nav__link--active" for="__toc">
How to use LargeFile-s
How to use LargeFiles
<span class="md-nav__icon md-icon"></span>
</label>
<a href="./" class="md-nav__link md-nav__link--active">
How to use LargeFile-s
How to use LargeFiles
</a>
@ -937,8 +937,8 @@
<h1 id="how-to-use-largefile-s">How to use LargeFile-s<a class="headerlink" href="#how-to-use-largefile-s" title="Permanent link">#</a></h1>
<p>The functions <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">save_model</a> and <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> wrap LargeFile instances. Hence, besides configuring LargeFile, users have few reasons to use LargeFile-s directly.</p>
<h1 id="how-to-use-largefiles">How to use LargeFiles<a class="headerlink" href="#how-to-use-largefiles" title="Permanent link">#</a></h1>
<p>The functions <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">save_model</a> and <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> wrap LargeFile instances. Hence, besides configuring <a href="/reference/large-file">LargeFile</a>, users have few reasons to use LargeFiles directly.</p>
<h2 id="motivation">Motivation<a class="headerlink" href="#motivation" title="Permanent link">#</a></h2>
<p>Oftentimes, especially when working with data-heavy applications, large files can proliferate in a repository. Version controlling them is an obvious next step, however, GitHub's git LFS implementation <a href="https://docs.github.com/en/repositories/working-with-files/managing-large-files/removing-files-from-git-large-file-storage#git-lfs-objects-in-your-repository">doesn't support deleting</a> large files, making it easy for them to eat-up the LFS quota and explode the size of your repos.</p>
<p><a href="https://dvc.org/">DVC</a> is a viable alternative, however, it requires users to learn to use one more CLI tool.</p>
@ -961,7 +961,7 @@
<a id="__codelineno-0-13" name="__codelineno-0-13" href="#__codelineno-0-13"></a> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100000</span><span class="p">):</span>
<a id="__codelineno-0-14" name="__codelineno-0-14" href="#__codelineno-0-14"></a> <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">&#39;test</span><span class="se">\n</span><span class="s1">&#39;</span><span class="p">)</span>
<a id="__codelineno-0-15" name="__codelineno-0-15" href="#__codelineno-0-15"></a>
<a id="__codelineno-0-16" name="__codelineno-0-16" href="#__codelineno-0-16"></a><span class="c1"># By default the latest version is returned</span>
<a id="__codelineno-0-16" name="__codelineno-0-16" href="#__codelineno-0-16"></a><span class="c1"># The latest version is returned by default</span>
<a id="__codelineno-0-17" name="__codelineno-0-17" href="#__codelineno-0-17"></a><span class="c1"># but an optional `version` keyword argument can be provided as well</span>
<a id="__codelineno-0-18" name="__codelineno-0-18" href="#__codelineno-0-18"></a><span class="k">with</span> <span class="n">LargeFileS3</span><span class="p">(</span><span class="s2">&quot;test.txt&quot;</span><span class="p">,</span> <span class="s2">&quot;r&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span> <span class="c1">#(1)</span>
<a id="__codelineno-0-19" name="__codelineno-0-19" href="#__codelineno-0-19"></a> <span class="nb">print</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">readlines</span><span class="p">()[</span><span class="mi">0</span><span class="p">])</span>
@ -970,13 +970,13 @@
<li>In this case, the latest version is already in the local cache, no download is required.</li>
</ol>
<h3 id="more-details">More details<a class="headerlink" href="#more-details" title="Permanent link">#</a></h3>
<p><code>LargeFile</code> behaves like an opened file (in the background it is a temp file after all). Binary reads and writes are supported along with the <a href="https://docs.python.org/3/library/functions.html#open">different keywords <code>open()</code> accepts</a>.</p>
<p><code>LargeFile</code> behaves like an opened file (in the background it is a temp file after all). Binary reads and writes are supported along with the <a href="https://docs.python.org/3/library/functions.html#open" target="_blank">different keywords <code>open()</code> accepts</a>.</p>
<p>The local cache can be configured with these properties:</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a><span class="n">LargeFileS3</span><span class="o">.</span><span class="n">cache_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s1">&#39;.cache&#39;</span><span class="p">)</span>
<a id="__codelineno-1-2" name="__codelineno-1-2" href="#__codelineno-1-2"></a><span class="n">LargeFileS3</span><span class="o">.</span><span class="n">max_cache_size</span> <span class="o">=</span> <span class="s2">&quot;30 GB&quot;</span>
</code></pre></div>
<h4 id="i-only-need-a-path">I only need a path<a class="headerlink" href="#i-only-need-a-path" title="Permanent link">#</a></h4>
<p>In case you only need a path to the "remote" file, this pattern can be applied:</p>
<p>In case you only need a path to the (proxy of the) remote file, this pattern can be applied:</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a><span class="n">path_to_model</span> <span class="o">=</span> <span class="n">LargeFileS3</span><span class="p">(</span><span class="s2">&quot;folder-of-my-bert-model&quot;</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="mi">31</span><span class="p">)</span><span class="o">.</span><span class="n">get</span><span class="p">()</span>
</code></pre></div>
<blockquote>
@ -988,7 +988,7 @@
<blockquote>
<p>This way both regular files and folders can be handled. The uploaded file is called <strong>folder-of-my-bert-model</strong>, the local name is ignored.</p>
</blockquote>
<p>Lastly, all version of the remote object can be deleted by calling <code>LargeFileS3("my-file").delete()</code>. It will still reside in your local cache afterwards, its deletion will happen next time your local cache has to be pruned.</p>
<p>Lastly, all version of the remote object can be deleted by calling <code>LargeFileS3("my-file").delete()</code>. It will still reside in your local cache afterwards; its deletion will happen next time your local cache has to be pruned.</p>
</details>
<h2 id="from-the-command-line">From the command-line<a class="headerlink" href="#from-the-command-line" title="Permanent link">#</a></h2>
<p>The main reason for using the <code>large-file</code> or <code>python3 -m great_ai.large_file</code> commands is to upload or download models from the terminal. For example, when building a docker image, it is best-practice to cache the referred models.</p>
@ -1008,7 +1008,7 @@
</blockquote>
<div class="admonition important">
<p class="admonition-title">Using MongoDB</p>
<p>The possible values for <code>--backend</code> are <code>s3</code>, <code>mongo</code>, and <code>local</code>. The latter doesn't need credentials, it only versions and stores your files in a local folder. MongoDB on the other hand requires a <code>mongo_connection_string</code> and a <code>mongo_database</code> to be specified. For storing large files, it uses the GridFS specification.</p>
<p>The possible values for <code>--backend</code> are <code>s3</code>, <code>mongo</code>, and <code>local</code>. The latter doesn't need credentials, it only versions and stores your files in a local folder. MongoDB on the other hand requires a <code>mongo_connection_string</code> and a <code>mongo_database</code> to be specified. For storing large files, it uses the <a href="https://www.mongodb.com/docs/manual/core/gridfs" target="_blank">GridFS</a> specification.</p>
</div>
<h3 id="download-some-files-to-the-local-cache">Download some files to the local cache<a class="headerlink" href="#download-some-files-to-the-local-cache" title="Permanent link">#</a></h3>
<p>This can be useful when building a Docker image for example. This way, the files can already reside inside the container and need not be downloaded later.</p>
@ -1028,7 +1028,7 @@
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 13, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
</small>

14
how-to-guides/large_file/large_file.md
View file

@ -1,6 +1,6 @@
# How to use LargeFile-s
# How to use LargeFiles
The functions [save_model][great_ai.use_model] and [@use_model][great_ai.use_model] wrap LargeFile instances. Hence, besides configuring LargeFile, users have few reasons to use LargeFile-s directly.
The functions [save_model][great_ai.use_model] and [@use_model][great_ai.use_model] wrap LargeFile instances. Hence, besides configuring [LargeFile](/reference/large-file), users have few reasons to use LargeFiles directly.
## Motivation
@ -29,7 +29,7 @@ Oftentimes, especially when working with data-heavy applications, large files ca
for i in range(100000):
f.write('test\n')
# By default the latest version is returned
# The latest version is returned by default
# but an optional `version` keyword argument can be provided as well
with LargeFileS3("test.txt", "r") as f: #(1)
print(f.readlines()[0])
@ -39,7 +39,7 @@ Oftentimes, especially when working with data-heavy applications, large files ca
### More details
`LargeFile` behaves like an opened file (in the background it is a temp file after all). Binary reads and writes are supported along with the [different keywords `open()` accepts](https://docs.python.org/3/library/functions.html#open).
`LargeFile` behaves like an opened file (in the background it is a temp file after all). Binary reads and writes are supported along with the [different keywords `open()` accepts](https://docs.python.org/3/library/functions.html#open){ target=_blank }.
The local cache can be configured with these properties:
@ -50,7 +50,7 @@ Oftentimes, especially when working with data-heavy applications, large files ca
#### I only need a path
In case you only need a path to the "remote" file, this pattern can be applied:
In case you only need a path to the (proxy of the) remote file, this pattern can be applied:
```python
path_to_model = LargeFileS3("folder-of-my-bert-model", version=31).get()
@ -66,7 +66,7 @@ Oftentimes, especially when working with data-heavy applications, large files ca
> This way both regular files and folders can be handled. The uploaded file is called **folder-of-my-bert-model**, the local name is ignored.
Lastly, all version of the remote object can be deleted by calling `LargeFileS3("my-file").delete()`. It will still reside in your local cache afterwards, its deletion will happen next time your local cache has to be pruned.
Lastly, all version of the remote object can be deleted by calling `LargeFileS3("my-file").delete()`. It will still reside in your local cache afterwards; its deletion will happen next time your local cache has to be pruned.
## From the command-line
@ -93,7 +93,7 @@ large-file --backend s3 --secrets secrets.ini \
> Only the filename is used as the S3 name, the rest of the path is ignored.
!!! important "Using MongoDB"
The possible values for `--backend` are `s3`, `mongo`, and `local`. The latter doesn't need credentials, it only versions and stores your files in a local folder. MongoDB on the other hand requires a `mongo_connection_string` and a `mongo_database` to be specified. For storing large files, it uses the GridFS specification.
The possible values for `--backend` are `s3`, `mongo`, and `local`. The latter doesn't need credentials, it only versions and stores your files in a local folder. MongoDB on the other hand requires a `mongo_connection_string` and a `mongo_database` to be specified. For storing large files, it uses the [GridFS](https://www.mongodb.com/docs/manual/core/gridfs){ target=_blank } specification.
### Download some files to the local cache

16
how-to-guides/use-service/index.html
View file

@ -540,7 +540,7 @@
<li class="md-nav__item">
<a href="../large_file/" class="md-nav__link">
How to use LargeFile-s
How to use LargeFiles
</a>
</li>
@ -918,7 +918,7 @@
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a>
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a><span class="nd">@GreatAI</span><span class="o">.</span><span class="n">create</span>
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a><span class="k">def</span> <span class="nf">greeter</span><span class="p">(</span><span class="n">your_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a> <span class="k">return</span> <span class="sa">f</span><span class="s1">&#39;Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s1">&#39;</span>
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a> <span class="k">return</span> <span class="sa">f</span><span class="s1">&#39;Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s1">!&#39;</span>
</code></pre></div>
<h2 id="one-off-prediction">One-off prediction<a class="headerlink" href="#one-off-prediction" title="Permanent link">#</a></h2>
<p>Even though <code>greeter</code> is now an instance of <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI</a>, you can continue using it as a regular function.</p>
@ -929,14 +929,14 @@
<a id="__codelineno-1-5" name="__codelineno-1-5" href="#__codelineno-1-5"></a> <span class="s1">&#39;logged_values&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;arg:your_name:length&#39;</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span> <span class="s1">&#39;arg:your_name:value&#39;</span><span class="p">:</span> <span class="s1">&#39;Bob&#39;</span><span class="p">},</span>
<a id="__codelineno-1-6" name="__codelineno-1-6" href="#__codelineno-1-6"></a> <span class="s1">&#39;models&#39;</span><span class="p">:</span> <span class="p">[],</span>
<a id="__codelineno-1-7" name="__codelineno-1-7" href="#__codelineno-1-7"></a> <span class="s1">&#39;original_execution_time_ms&#39;</span><span class="p">:</span> <span class="mf">0.0381</span><span class="p">,</span>
<a id="__codelineno-1-8" name="__codelineno-1-8" href="#__codelineno-1-8"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Bob&#39;</span><span class="p">,</span>
<a id="__codelineno-1-8" name="__codelineno-1-8" href="#__codelineno-1-8"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Bob!&#39;</span><span class="p">,</span>
<a id="__codelineno-1-9" name="__codelineno-1-9" href="#__codelineno-1-9"></a> <span class="s1">&#39;tags&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;greeter&#39;</span><span class="p">,</span> <span class="s1">&#39;online&#39;</span><span class="p">,</span> <span class="s1">&#39;development&#39;</span><span class="p">],</span>
<a id="__codelineno-1-10" name="__codelineno-1-10" href="#__codelineno-1-10"></a> <span class="s1">&#39;trace_id&#39;</span><span class="p">:</span> <span class="s1">&#39;7c284fd7-7f0d-4464-b5f8-3ef126df34af&#39;</span><span class="p">})</span>
</code></pre></div>
<p>As you can see, the original return value is wrapped in a <a class="autorefs autorefs-internal" href="../../reference/views/#great_ai.Trace">Trace</a> object (which is also persisted in your database of choice). You can access the original value under the <code>output</code> property.</p>
<h2 id="online-prediction">Online prediction<a class="headerlink" href="#online-prediction" title="Permanent link">#</a></h2>
<p>Likely, the main way you would like to expose your model is through an HTTP API. <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> scaffolds many REST API endpoints for your model and creates a <a href="https://fastapi.tiangolo.com/" target="_blank">FastAPI</a> app available under <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI.app</a>. This can be served using <a href="https://www.uvicorn.org/" target="_blank">uvicorn</a> or any other <a href="https://asgi.readthedocs.io/en/latest/" target="_blank">ASGI server</a>.</p>
<p>Since most ML code lives in <a href="https://jupyter.org/" target="_blank">Jupyter</a> notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, <code>uvicorn</code> is wrapped by the <code>great-ai</code> command-line utility which, among others, takes care of feeding a notebook into <code>uvicorn</code>. It also supports auto-reloading.</p>
<p>Since most ML code lives in <a href="https://jupyter.org/" target="_blank">Jupyter</a> notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, <code>uvicorn</code> is wrapped by the <code>great-ai</code> command-line utility which &mdash; among others &mdash; takes care of feeding a notebook into <code>uvicorn</code>. It also supports auto-reloading.</p>
<h3 id="in-development">In development<a class="headerlink" href="#in-development" title="Permanent link">#</a></h3>
<div class="highlight"><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a>great-ai greeter.py
</code></pre></div>
@ -959,7 +959,7 @@
</code></pre></div>
<p>Simply run <code>ENVIRONMENT=production great-ai deploy.ipynb</code> in the command-line of a production machine.</p>
<blockquote>
<p>This is the crudest approach, however, it might be fitting for some contexts.</p>
<p>This is the crudest approach; however, it might be fitting for some contexts.</p>
</blockquote>
<h4 id="containerised-deployment">Containerised deployment<a class="headerlink" href="#containerised-deployment" title="Permanent link">#</a></h4>
<p>Run the notebook directly in a container or create a service for it using your favourite container orchestrator.</p>
@ -970,7 +970,7 @@
<p>You can replace <code>pwd</code> with the path to your code's folder.</p>
</blockquote>
<h4 id="use-a-platform-as-a-service">Use a Platform-as-a-Service<a class="headerlink" href="#use-a-platform-as-a-service" title="Permanent link">#</a></h4>
<p>Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as <a href="https://aws.amazon.com/ecs/" target="_blank">AWS ECS</a>, <a href="https://www.digitalocean.com/products/app-platform" target="_blank">DO App platform</a>, <a href="https://mlem.ai/" target="_blank">MLEM</a>, <a href="https://streamlit.io/" target="_blank">Streamlit</a>) that take a Docker image as a source and handle the rest of the deployment.</p>
<p>Similar to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as <a href="https://aws.amazon.com/ecs/" target="_blank">AWS ECS</a>, <a href="https://www.digitalocean.com/products/app-platform" target="_blank">DO App platform</a>, <a href="https://mlem.ai/" target="_blank">MLEM</a>, <a href="https://streamlit.io/" target="_blank">Streamlit</a>) that take a Docker image as a source and handle the rest of the deployment.</p>
<p>To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as <a href="https://pytorch.org/" target="_blank">PyTorch</a> or <a href="https://www.tensorflow.org/" target="_blank">TensorFlow</a>.</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a><span class="k">FROM</span><span class="w"> </span><span class="s">schmelczera/great-ai:latest</span>
<a id="__codelineno-6-2" name="__codelineno-6-2" href="#__codelineno-6-2"></a>
@ -998,7 +998,7 @@
<a id="__codelineno-7-5" name="__codelineno-7-5" href="#__codelineno-7-5"></a> <span class="s1">&#39;logged_values&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;arg:your_name:length&#39;</span><span class="p">:</span> <span class="mi">5</span><span class="p">,</span> <span class="s1">&#39;arg:your_name:value&#39;</span><span class="p">:</span> <span class="s1">&#39;Alice&#39;</span><span class="p">},</span>
<a id="__codelineno-7-6" name="__codelineno-7-6" href="#__codelineno-7-6"></a> <span class="s1">&#39;models&#39;</span><span class="p">:</span> <span class="p">[],</span>
<a id="__codelineno-7-7" name="__codelineno-7-7" href="#__codelineno-7-7"></a> <span class="s1">&#39;original_execution_time_ms&#39;</span><span class="p">:</span> <span class="mf">0.1251</span><span class="p">,</span>
<a id="__codelineno-7-8" name="__codelineno-7-8" href="#__codelineno-7-8"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Alice&#39;</span><span class="p">,</span>
<a id="__codelineno-7-8" name="__codelineno-7-8" href="#__codelineno-7-8"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Alice!&#39;</span><span class="p">,</span>
<a id="__codelineno-7-9" name="__codelineno-7-9" href="#__codelineno-7-9"></a> <span class="s1">&#39;tags&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;greeter&#39;</span><span class="p">,</span> <span class="s1">&#39;online&#39;</span><span class="p">,</span> <span class="s1">&#39;development&#39;</span><span class="p">],</span>
<a id="__codelineno-7-10" name="__codelineno-7-10" href="#__codelineno-7-10"></a> <span class="s1">&#39;trace_id&#39;</span><span class="p">:</span> <span class="s1">&#39;90ffa15f-e839-41c4-8e7a-3211168bc138&#39;</span><span class="p">}),</span>
<a id="__codelineno-7-11" name="__codelineno-7-11" href="#__codelineno-7-11"></a> <span class="n">Trace</span><span class="p">[</span><span class="nb">str</span><span class="p">]({</span><span class="s1">&#39;created&#39;</span><span class="p">:</span> <span class="s1">&#39;2022-07-11T14:36:37.166659&#39;</span><span class="p">,</span>
@ -1007,7 +1007,7 @@
<a id="__codelineno-7-14" name="__codelineno-7-14" href="#__codelineno-7-14"></a> <span class="s1">&#39;logged_values&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;arg:your_name:length&#39;</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span> <span class="s1">&#39;arg:your_name:value&#39;</span><span class="p">:</span> <span class="s1">&#39;Bob&#39;</span><span class="p">},</span>
<a id="__codelineno-7-15" name="__codelineno-7-15" href="#__codelineno-7-15"></a> <span class="s1">&#39;models&#39;</span><span class="p">:</span> <span class="p">[],</span>
<a id="__codelineno-7-16" name="__codelineno-7-16" href="#__codelineno-7-16"></a> <span class="s1">&#39;original_execution_time_ms&#39;</span><span class="p">:</span> <span class="mf">0.0571</span><span class="p">,</span>
<a id="__codelineno-7-17" name="__codelineno-7-17" href="#__codelineno-7-17"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Bob&#39;</span><span class="p">,</span>
<a id="__codelineno-7-17" name="__codelineno-7-17" href="#__codelineno-7-17"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Bob!&#39;</span><span class="p">,</span>
<a id="__codelineno-7-18" name="__codelineno-7-18" href="#__codelineno-7-18"></a> <span class="s1">&#39;tags&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;greeter&#39;</span><span class="p">,</span> <span class="s1">&#39;online&#39;</span><span class="p">,</span> <span class="s1">&#39;development&#39;</span><span class="p">],</span>
<a id="__codelineno-7-19" name="__codelineno-7-19" href="#__codelineno-7-19"></a> <span class="s1">&#39;trace_id&#39;</span><span class="p">:</span> <span class="s1">&#39;f48e94c7-0815-48b3-a864-41349d3dae84&#39;</span><span class="p">})]</span>
</code></pre></div>

14
how-to-guides/use-service/use-service.md
View file

@ -9,7 +9,7 @@ from great_ai import GreatAI
@GreatAI.create
def greeter(your_name: str) -> str:
return f'Hi {your_name}'
return f'Hi {your_name}!'
```
## One-off prediction
@ -24,7 +24,7 @@ Trace[str]({'created': '2022-07-11T14:31:46.183764',
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
'models': [],
'original_execution_time_ms': 0.0381,
'output': 'Hi Bob',
'output': 'Hi Bob!',
'tags': ['greeter', 'online', 'development'],
'trace_id': '7c284fd7-7f0d-4464-b5f8-3ef126df34af'})
```
@ -35,7 +35,7 @@ As you can see, the original return value is wrapped in a [Trace][great_ai.Trace
Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, `uvicorn` is wrapped by the `great-ai` command-line utility which, among others, takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, `uvicorn` is wrapped by the `great-ai` command-line utility which &mdash; among others &mdash; takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
### In development
@ -67,7 +67,7 @@ ENVIRONMENT=production great-ai greeter.py
```
Simply run `ENVIRONMENT=production great-ai deploy.ipynb` in the command-line of a production machine.
> This is the crudest approach, however, it might be fitting for some contexts.
> This is the crudest approach; however, it might be fitting for some contexts.
#### Containerised deployment
@ -81,7 +81,7 @@ docker run -p 6060:6060 --volume `pwd`:/app --rm \
#### Use a Platform-as-a-Service
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [AWS ECS](https://aws.amazon.com/ecs/){ target=_blank }, [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank }, [MLEM](https://mlem.ai/){ target=_blank }, [Streamlit](https://streamlit.io/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
Similar to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [AWS ECS](https://aws.amazon.com/ecs/){ target=_blank }, [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank }, [MLEM](https://mlem.ai/){ target=_blank }, [Streamlit](https://streamlit.io/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.
@ -117,7 +117,7 @@ Processing larger amounts of data on a single machine is made easy by the [Great
'logged_values': {'arg:your_name:length': 5, 'arg:your_name:value': 'Alice'},
'models': [],
'original_execution_time_ms': 0.1251,
'output': 'Hi Alice',
'output': 'Hi Alice!',
'tags': ['greeter', 'online', 'development'],
'trace_id': '90ffa15f-e839-41c4-8e7a-3211168bc138'}),
Trace[str]({'created': '2022-07-11T14:36:37.166659',
@ -126,7 +126,7 @@ Processing larger amounts of data on a single machine is made easy by the [Great
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
'models': [],
'original_execution_time_ms': 0.0571,
'output': 'Hi Bob',
'output': 'Hi Bob!',
'tags': ['greeter', 'online', 'development'],
'trace_id': 'f48e94c7-0815-48b3-a864-41349d3dae84'})]
```