Deployed b5a69fe with MkDocs version: 1.3.1
This commit is contained in:
parent
3cf6b08552
commit
fbc107f453
25 changed files with 169 additions and 170 deletions
|
|
@ -989,23 +989,23 @@
|
|||
|
||||
|
||||
<h1 id="summarising-scientific-publications-from-a-tech-transfer-perspective">Summarising scientific publications from a tech-transfer perspective<a class="headerlink" href="#summarising-scientific-publications-from-a-tech-transfer-perspective" title="Permanent link">#</a></h1>
|
||||
<p>This is a simplified example illustrating how <code>great-ai</code> is used in practice at <a href="https://www.scoutinscience.com/" target="_blank">ScoutinScience</a>. The subpages show <code>great-ai</code> in action by going over the life-cycle of finetuning and deploying a BERT-based software service.</p>
|
||||
<p>This is a simplified example illustrating how <code>great-ai</code> is used in practice at <a href="https://www.scoutinscience.com/" target="_blank">ScoutinScience</a>. The subpages show <code>great-ai</code> in action by going over the lifecycle of fine-tuning and deploying a BERT-based software service.</p>
|
||||
<details class="note">
|
||||
<summary>Propriety data</summary>
|
||||
<p>The purpose of this example is to show you different ways in which <code>great-ai</code> can assist you. The exact NLP task being solved is not central. Stemming from this, and from the difficult nature of obtaining appropriate training data, the propriety dataset used for the experiments is not shared.</p>
|
||||
<p>The purpose of this example is to show you different ways in which <code>great-ai</code> can assist you. The exact NLP task being solved is not central. Stemming from this and from the difficult nature of obtaining appropriate training data, the propriety dataset used for the experiments is not shared.</p>
|
||||
</details>
|
||||
<h2 id="objectives">Objectives<a class="headerlink" href="#objectives" title="Permanent link">#</a></h2>
|
||||
<ol>
|
||||
<li>You will see how the <a href="/reference/utilities">great_ai.utilities</a> can integrate into your Data Science workflow.</li>
|
||||
<li>You will see how <a href="/reference/large-file">great_ai.large_file</a> can be used to version and store your trained model.</li>
|
||||
<li>You will see how <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI</a> should be used prepare your model for a robust and responsible deployment.</li>
|
||||
<li>You will see how <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI</a> should be used to prepare your model for a robust and responsible deployment.</li>
|
||||
<li>You will see multiple ways of customising your deployment.</li>
|
||||
</ol>
|
||||
<h2 id="overview">Overview<a class="headerlink" href="#overview" title="Permanent link">#</a></h2>
|
||||
<p>One of the core features of the ScoutinScience platform is summarising research papers from a tech-transfer perspective. In short, extractive summarisation is preferred using a binary classifier trained on clients' judgement of sentence interestingness. Thus, documents are sentences and the expected output is a binary label showing whether a sentence is "worthy" of being in the tech-transfer summary. Providing an explanation for each decision is imperative since ScoutinScience embraces applying only explainable AI (XAI) methods wherever feasible.</p>
|
||||
<p>One of the core features of the ScoutinScience platform is summarising research papers from a tech-transfer perspective. In short, extractive summarisation is preferred using a binary classifier trained on clients' judgement of sentence interestingness. Thus, documents are sentences, and the expected output is a binary label showing whether a sentence is "worthy" of being in the tech-transfer summary. Explaining each decision is imperative since ScoutinScience embraces applying only explainable AI (XAI) methods wherever feasible.</p>
|
||||
<div class="admonition success">
|
||||
<p class="admonition-title">Success</p>
|
||||
<p>You are ready to start the tutorial. Feel free to come back to the <a href="#summary">summary</a> section once you're finished.</p>
|
||||
<p>You are ready to start the tutorial. Feel free to return to the <a href="#summary">summary</a> section once you're finished.</p>
|
||||
</div>
|
||||
<div style="display: flex; justify-content: space-evenly;">
|
||||
<p><a class="md-button md-button--primary" href="data/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 3C7.58 3 4 4.79 4 7s3.58 4 8 4 8-1.79 8-4-3.58-4-8-4M4 9v3c0 2.21 3.58 4 8 4s8-1.79 8-4V9c0 2.21-3.58 4-8 4s-8-1.79-8-4m0 5v3c0 2.21 3.58 4 8 4s8-1.79 8-4v-3c0 2.21-3.58 4-8 4s-8-1.79-8-4Z"/></svg></span> Examine data</a></p>
|
||||
|
|
@ -1019,17 +1019,17 @@
|
|||
<p>We simply fine-tune SciBERT.</p>
|
||||
<p>After training and evaluating a model, it is exported using <a class="autorefs autorefs-internal" href="../../reference/#great_ai.save_model">great_ai.save_model</a>. For more info, check out <a href="/how-to-guides/configure-service">the configuration how-to page</a>.</p>
|
||||
<h3 id="deployment-notebook"><a href="deploy/">Deployment notebook</a><a class="headerlink" href="#deployment-notebook" title="Permanent link">#</a></h3>
|
||||
<p>We customise the GreatAI configuration, create custom cahcing for the model and implement an inference function that can be hardened by wrapping it in a <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI</a> instance. We also extract the attention weights as a quasi-explanation.</p>
|
||||
<p>We customise the GreatAI configuration, create custom caching for the model and implement an inference function that can be hardened by wrapping it in a <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI</a> instance. We also extract the attention weights as a quasi-explanation.</p>
|
||||
<p>Finally, we test the model's inference function through the GreatAI dashboard. <a href="/how-to-guides/use-service">The only thing left is to deploy the hardened service properly.</a></p>
|
||||
<h4 id="additional-files"><a href="additional-files/">Additional files</a><a class="headerlink" href="#additional-files" title="Permanent link">#</a></h4>
|
||||
<p>There are some other files required for deploying the notebook. For example, the config file for S3 and MongoDB or a Dockerfile for building a custom image. These are compiled and shown on a separate page.</p>
|
||||
<p>There are some other files required for deploying the notebook. For example, the config file for S3 and MongoDB or a Dockerfile for building a custom image. These are gathered and shown on a separate page.</p>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 17, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -1,23 +1,23 @@
|
|||
# Summarising scientific publications from a tech-transfer perspective
|
||||
|
||||
This is a simplified example illustrating how `great-ai` is used in practice at [ScoutinScience](https://www.scoutinscience.com/){ target=_blank }. The subpages show `great-ai` in action by going over the life-cycle of finetuning and deploying a BERT-based software service.
|
||||
This is a simplified example illustrating how `great-ai` is used in practice at [ScoutinScience](https://www.scoutinscience.com/){ target=_blank }. The subpages show `great-ai` in action by going over the lifecycle of fine-tuning and deploying a BERT-based software service.
|
||||
|
||||
??? note "Propriety data"
|
||||
The purpose of this example is to show you different ways in which `great-ai` can assist you. The exact NLP task being solved is not central. Stemming from this, and from the difficult nature of obtaining appropriate training data, the propriety dataset used for the experiments is not shared.
|
||||
The purpose of this example is to show you different ways in which `great-ai` can assist you. The exact NLP task being solved is not central. Stemming from this and from the difficult nature of obtaining appropriate training data, the propriety dataset used for the experiments is not shared.
|
||||
|
||||
## Objectives
|
||||
|
||||
1. You will see how the [great_ai.utilities](/reference/utilities) can integrate into your Data Science workflow.
|
||||
2. You will see how [great_ai.large_file](/reference/large-file) can be used to version and store your trained model.
|
||||
3. You will see how [GreatAI][great_ai.GreatAI] should be used prepare your model for a robust and responsible deployment.
|
||||
3. You will see how [GreatAI][great_ai.GreatAI] should be used to prepare your model for a robust and responsible deployment.
|
||||
4. You will see multiple ways of customising your deployment.
|
||||
|
||||
## Overview
|
||||
|
||||
One of the core features of the ScoutinScience platform is summarising research papers from a tech-transfer perspective. In short, extractive summarisation is preferred using a binary classifier trained on clients' judgement of sentence interestingness. Thus, documents are sentences and the expected output is a binary label showing whether a sentence is "worthy" of being in the tech-transfer summary. Providing an explanation for each decision is imperative since ScoutinScience embraces applying only explainable AI (XAI) methods wherever feasible.
|
||||
One of the core features of the ScoutinScience platform is summarising research papers from a tech-transfer perspective. In short, extractive summarisation is preferred using a binary classifier trained on clients' judgement of sentence interestingness. Thus, documents are sentences, and the expected output is a binary label showing whether a sentence is "worthy" of being in the tech-transfer summary. Explaining each decision is imperative since ScoutinScience embraces applying only explainable AI (XAI) methods wherever feasible.
|
||||
|
||||
!!! success
|
||||
You are ready to start the tutorial. Feel free to come back to the [summary](#summary) section once you're finished.
|
||||
You are ready to start the tutorial. Feel free to return to the [summary](#summary) section once you're finished.
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||
[:material-database: Examine data](data.ipynb){ .md-button .md-button--primary }
|
||||
|
|
@ -41,10 +41,10 @@ After training and evaluating a model, it is exported using [great_ai.save_model
|
|||
|
||||
### [Deployment notebook](deploy.ipynb)
|
||||
|
||||
We customise the GreatAI configuration, create custom cahcing for the model and implement an inference function that can be hardened by wrapping it in a [GreatAI][great_ai.GreatAI] instance. We also extract the attention weights as a quasi-explanation.
|
||||
We customise the GreatAI configuration, create custom caching for the model and implement an inference function that can be hardened by wrapping it in a [GreatAI][great_ai.GreatAI] instance. We also extract the attention weights as a quasi-explanation.
|
||||
|
||||
Finally, we test the model's inference function through the GreatAI dashboard. [The only thing left is to deploy the hardened service properly.](/how-to-guides/use-service)
|
||||
|
||||
#### [Additional files](additional-files.md)
|
||||
|
||||
There are some other files required for deploying the notebook. For example, the config file for S3 and MongoDB or a Dockerfile for building a custom image. These are compiled and shown on a separate page.
|
||||
There are some other files required for deploying the notebook. For example, the config file for S3 and MongoDB or a Dockerfile for building a custom image. These are gathered and shown on a separate page.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Explanation
|
||||
|
||||
A lot more details and discussion about the problem context and approaches of GreatAI along with its evaluation can be found in my thesis.
|
||||
A lot more details and discussion about the problem context and approaches of GreatAI, along with its evaluation, can be found in my thesis.
|
||||
|
||||
<div style="display: flex; justify-content: center;">
|
||||
<img src="/media/thesis-frontpage.png" style="height: 500px;" alt="front page"/>
|
||||
|
|
|
|||
|
|
@ -831,7 +831,7 @@
|
|||
|
||||
|
||||
<h1 id="explanation">Explanation<a class="headerlink" href="#explanation" title="Permanent link">#</a></h1>
|
||||
<p>A lot more details and discussion about the problem context and approaches of GreatAI along with its evaluation can be found in my thesis.</p>
|
||||
<p>A lot more details and discussion about the problem context and approaches of GreatAI, along with its evaluation, can be found in my thesis.</p>
|
||||
<div style="display: flex; justify-content: center;">
|
||||
<img src="/media/thesis-frontpage.png" style="height: 500px;" alt="front page"/>
|
||||
</div>
|
||||
|
|
@ -845,7 +845,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 6, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# How to call remote GreatAI instances
|
||||
|
||||
Microservices architecture (or [SOA](https://en.wikipedia.org/wiki/Service-oriented_architecture)) work well with ML applications. This is because their interfaces are usually very narrow, while the functionality provided quite comprehensive. Hence, drawing the boundaries of responsibilities is more straightforward in the case of ML services than in the case of more traditional business applications. For this reason, it is common to have a tree of models (preferably wrapped in GreatAI instances) communicating with each other.
|
||||
Microservices architecture (or [SOA](https://en.wikipedia.org/wiki/Service-oriented_architecture)) work well with ML applications. This is because their interfaces are usually very narrow, while the functionality provided is quite comprehensive. Hence, drawing the boundaries of responsibilities is more straightforward in the case of ML services than in the case of more traditional business applications. For this reason, it is common to have a tree of models (preferably wrapped in GreatAI instances) communicating with each other.
|
||||
|
||||
Although regular HTTP POST requests could be sent to each service's `/predict` endpoint, `great-ai` comes with two convenience functions: [call_remote_great_ai][great_ai.call_remote_great_ai] and [call_remote_great_ai_async][great_ai.call_remote_great_ai_async] to wrap this request. These provide you with some level of robustness and deserialisation.
|
||||
|
||||
|
|
@ -44,21 +44,21 @@ results = [
|
|||
print(results)
|
||||
```
|
||||
|
||||
1. Only return the outputs so we don't clutter up the terminal.
|
||||
1. Only return the outputs, so we don't clutter up the terminal.
|
||||
|
||||
> Run this script as a regular Python script by executing `python3 client.py`.
|
||||
|
||||
{ loading=lazy }
|
||||
|
||||
As you can see, everything worked as expected. There is one way to improve it though.
|
||||
As you can see, everything worked as expected. There is one way to improve it, though.
|
||||
## An `async` example
|
||||
|
||||
Let's send multiple requests at the same time to speed up the overall execution time. To do this, we will use the [call_remote_great_ai_async][great_ai.call_remote_great_ai_async] function.
|
||||
Let's send multiple requests simultaneously to speed up the overall execution time. To do this, we will use the [call_remote_great_ai_async][great_ai.call_remote_great_ai_async] function.
|
||||
|
||||
??? note "Why is this possible?"
|
||||
Note, that in `server.py`, the inference function is declared `async`. This means that multiple "copies" of it can run at the same time in the same thread. Since, there is no CPU bottleneck, the server has a quite large throughput (requests responded to per second), but its latency will stay around 2 seconds due to the async `sleep` command.
|
||||
Note that in `server.py`, the inference function is declared `async`. This means that multiple "copies" of it can run at the same time in the same thread. Since there is no CPU bottleneck, the server has a quite large throughput (requests responded to per second), but its latency will stay around 2 seconds due to the async `sleep` command.
|
||||
|
||||
If your great-ai server is not `async`, higher throughput can be achieved by running multiple instances of it, either manually, or by running it with multiple `uvicorn` workers like this: `ENVIRONMENT=production great-ai server.py --worker_count 4`
|
||||
If your great-ai server is not `async`, higher throughput can be achieved by running multiple instances of it, either manually or by running it with multiple `uvicorn` workers like this: `ENVIRONMENT=production great-ai server.py --worker_count 4`
|
||||
|
||||
### Async client
|
||||
|
||||
|
|
@ -84,8 +84,8 @@ async def main():
|
|||
asyncio.run(main())
|
||||
```
|
||||
|
||||
> Replace `client.py` with this async client. Note that even though async support is significantly more streamlined in recent Python versions, it still requires a bit more boilerplate than its synchronous counterpart.
|
||||
> Replace `client.py` with this async client. Note that although async support is significantly more streamlined in recent Python versions, it still requires a bit more boilerplate than its synchronous counterpart.
|
||||
|
||||
{ loading=lazy }
|
||||
|
||||
This also works, and in some use-cases might be considerably quicker.
|
||||
This also works and might be considerably quicker in some use cases.
|
||||
|
|
|
|||
|
|
@ -959,7 +959,7 @@
|
|||
|
||||
|
||||
<h1 id="how-to-call-remote-greatai-instances">How to call remote GreatAI instances<a class="headerlink" href="#how-to-call-remote-greatai-instances" title="Permanent link">#</a></h1>
|
||||
<p>Microservices architecture (or <a href="https://en.wikipedia.org/wiki/Service-oriented_architecture">SOA</a>) work well with ML applications. This is because their interfaces are usually very narrow, while the functionality provided quite comprehensive. Hence, drawing the boundaries of responsibilities is more straightforward in the case of ML services than in the case of more traditional business applications. For this reason, it is common to have a tree of models (preferably wrapped in GreatAI instances) communicating with each other.</p>
|
||||
<p>Microservices architecture (or <a href="https://en.wikipedia.org/wiki/Service-oriented_architecture">SOA</a>) work well with ML applications. This is because their interfaces are usually very narrow, while the functionality provided is quite comprehensive. Hence, drawing the boundaries of responsibilities is more straightforward in the case of ML services than in the case of more traditional business applications. For this reason, it is common to have a tree of models (preferably wrapped in GreatAI instances) communicating with each other.</p>
|
||||
<p>Although regular HTTP POST requests could be sent to each service's <code>/predict</code> endpoint, <code>great-ai</code> comes with two convenience functions: <a class="autorefs autorefs-internal" href="../../reference/#great_ai.call_remote_great_ai">call_remote_great_ai</a> and <a class="autorefs autorefs-internal" href="../../reference/#great_ai.call_remote_great_ai_async">call_remote_great_ai_async</a> to wrap this request. These provide you with some level of robustness and deserialisation.</p>
|
||||
<div class="admonition note">
|
||||
<p class="admonition-title">Inside notebooks</p>
|
||||
|
|
@ -997,19 +997,19 @@
|
|||
<a id="__codelineno-1-15" name="__codelineno-1-15" href="#__codelineno-1-15"></a><span class="nb">print</span><span class="p">(</span><span class="n">results</span><span class="p">)</span>
|
||||
</code></pre></div>
|
||||
<ol>
|
||||
<li>Only return the outputs so we don't clutter up the terminal.</li>
|
||||
<li>Only return the outputs, so we don't clutter up the terminal.</li>
|
||||
</ol>
|
||||
<blockquote>
|
||||
<p>Run this script as a regular Python script by executing <code>python3 client.py</code>.</p>
|
||||
</blockquote>
|
||||
<p><img alt="screenshot of result" loading="lazy" src="/media/remote-sync.png" /></p>
|
||||
<p>As you can see, everything worked as expected. There is one way to improve it though.</p>
|
||||
<p>As you can see, everything worked as expected. There is one way to improve it, though.</p>
|
||||
<h2 id="an-async-example">An <code>async</code> example<a class="headerlink" href="#an-async-example" title="Permanent link">#</a></h2>
|
||||
<p>Let's send multiple requests at the same time to speed up the overall execution time. To do this, we will use the <a class="autorefs autorefs-internal" href="../../reference/#great_ai.call_remote_great_ai_async">call_remote_great_ai_async</a> function.</p>
|
||||
<p>Let's send multiple requests simultaneously to speed up the overall execution time. To do this, we will use the <a class="autorefs autorefs-internal" href="../../reference/#great_ai.call_remote_great_ai_async">call_remote_great_ai_async</a> function.</p>
|
||||
<details class="note">
|
||||
<summary>Why is this possible?</summary>
|
||||
<p>Note, that in <code>server.py</code>, the inference function is declared <code>async</code>. This means that multiple "copies" of it can run at the same time in the same thread. Since, there is no CPU bottleneck, the server has a quite large throughput (requests responded to per second), but its latency will stay around 2 seconds due to the async <code>sleep</code> command.</p>
|
||||
<p>If your great-ai server is not <code>async</code>, higher throughput can be achieved by running multiple instances of it, either manually, or by running it with multiple <code>uvicorn</code> workers like this: <code>ENVIRONMENT=production great-ai server.py --worker_count 4</code></p>
|
||||
<p>Note that in <code>server.py</code>, the inference function is declared <code>async</code>. This means that multiple "copies" of it can run at the same time in the same thread. Since there is no CPU bottleneck, the server has a quite large throughput (requests responded to per second), but its latency will stay around 2 seconds due to the async <code>sleep</code> command.</p>
|
||||
<p>If your great-ai server is not <code>async</code>, higher throughput can be achieved by running multiple instances of it, either manually or by running it with multiple <code>uvicorn</code> workers like this: <code>ENVIRONMENT=production great-ai server.py --worker_count 4</code></p>
|
||||
</details>
|
||||
<h3 id="async-client">Async client<a class="headerlink" href="#async-client" title="Permanent link">#</a></h3>
|
||||
<div class="highlight"><span class="filename">async-client.py</span><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">call_remote_great_ai_async</span>
|
||||
|
|
@ -1033,17 +1033,17 @@
|
|||
<a id="__codelineno-2-19" name="__codelineno-2-19" href="#__codelineno-2-19"></a><span class="n">asyncio</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">main</span><span class="p">())</span>
|
||||
</code></pre></div>
|
||||
<blockquote>
|
||||
<p>Replace <code>client.py</code> with this async client. Note that even though async support is significantly more streamlined in recent Python versions, it still requires a bit more boilerplate than its synchronous counterpart.</p>
|
||||
<p>Replace <code>client.py</code> with this async client. Note that although async support is significantly more streamlined in recent Python versions, it still requires a bit more boilerplate than its synchronous counterpart.</p>
|
||||
</blockquote>
|
||||
<p><img alt="screenshot of result" loading="lazy" src="/media/remote-async.png" /></p>
|
||||
<p>This also works, and in some use-cases might be considerably quicker.</p>
|
||||
<p>This also works and might be considerably quicker in some use cases.</p>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ MONGO_CONNECTION_STRING=mongodb://localhost:27017 # this is the default value
|
|||
# if `MONGO_CONNECTION_STRING` is specified, this default is overridden
|
||||
MONGO_CONNECTION_STRING=ENV:MONGO_CONNECTION_STRING
|
||||
|
||||
MONGO_DATABASE=my-database # it is automatically created if doesn't exist
|
||||
MONGO_DATABASE=my-database # it is automatically created if it doesn't exist
|
||||
```
|
||||
|
||||
```python title="use-mongo.py"
|
||||
|
|
@ -99,4 +99,4 @@ By default, a thread-safe version of [TinyDB](https://tinydb.readthedocs.io/en/l
|
|||
|
||||
### MongoDB
|
||||
|
||||
At the moment, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory, or explicitly call either [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].
|
||||
Currently, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory or explicitly call either [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].
|
||||
|
|
|
|||
|
|
@ -1037,7 +1037,7 @@
|
|||
<a id="__codelineno-3-2" name="__codelineno-3-2" href="#__codelineno-3-2"></a><span class="c1"># if `MONGO_CONNECTION_STRING` is specified, this default is overridden</span><span class="w"></span>
|
||||
<a id="__codelineno-3-3" name="__codelineno-3-3" href="#__codelineno-3-3"></a><span class="n">MONGO_CONNECTION_STRING</span><span class="o">=</span><span class="n">ENV</span><span class="p">:</span><span class="n">MONGO_CONNECTION_STRING</span><span class="w"></span>
|
||||
<a id="__codelineno-3-4" name="__codelineno-3-4" href="#__codelineno-3-4"></a>
|
||||
<a id="__codelineno-3-5" name="__codelineno-3-5" href="#__codelineno-3-5"></a><span class="n">MONGO_DATABASE</span><span class="o">=</span><span class="n">my-database</span><span class="w"> </span><span class="c1"># it is automatically created if doesn't exist</span><span class="w"></span>
|
||||
<a id="__codelineno-3-5" name="__codelineno-3-5" href="#__codelineno-3-5"></a><span class="n">MONGO_DATABASE</span><span class="o">=</span><span class="n">my-database</span><span class="w"> </span><span class="c1"># it is automatically created if it doesn't exist</span><span class="w"></span>
|
||||
</code></pre></div>
|
||||
<div class="highlight"><span class="filename">use-mongo.py</span><pre><span></span><code><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="kn">from</span> <span class="nn">great_ai.large_file</span> <span class="kn">import</span> <span class="n">LargeFileMongo</span>
|
||||
<a id="__codelineno-4-2" name="__codelineno-4-2" href="#__codelineno-4-2"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">save_model</span>
|
||||
|
|
@ -1054,14 +1054,14 @@
|
|||
<h2 id="using-a-database">Using a database<a class="headerlink" href="#using-a-database" title="Permanent link">#</a></h2>
|
||||
<p>By default, a thread-safe version of <a href="https://tinydb.readthedocs.io/en/latest/" target="_blank">TinyDB</a> is utilised for saving the prediction traces into a local file. Unfortunately, for most production needs, this method is not suitable.</p>
|
||||
<h3 id="mongodb">MongoDB<a class="headerlink" href="#mongodb" title="Permanent link">#</a></h3>
|
||||
<p>At the moment, only MongoDB is supported as a production-ready <code>TracingDatabase</code>. In order to use it, you have to either place a file named <code>mongo.ini</code> in your working directory, or explicitly call either <a class="autorefs autorefs-internal" href="../../reference/#great_ai.MongoDbDriver">MongoDbDriver.configure_credentials_from_file</a> or <a class="autorefs autorefs-internal" href="../../reference/#great_ai.persistence.mongodb_driver.MongoDbDriver.configure_credentials">MongoDbDriver.configure_credentials</a>.</p>
|
||||
<p>Currently, only MongoDB is supported as a production-ready <code>TracingDatabase</code>. In order to use it, you have to either place a file named <code>mongo.ini</code> in your working directory or explicitly call either <a class="autorefs autorefs-internal" href="../../reference/#great_ai.MongoDbDriver">MongoDbDriver.configure_credentials_from_file</a> or <a class="autorefs autorefs-internal" href="../../reference/#great_ai.persistence.mongodb_driver.MongoDbDriver.configure_credentials">MongoDbDriver.configure_credentials</a>.</p>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# How to create a GreatAI service
|
||||
|
||||
The core value of `great-ai` lies in its [GreatAI][great_ai.GreatAI] class. In order to take advantage of it, you need to create an instance wrapping your code.
|
||||
The core value of `great-ai` lies in its [GreatAI][great_ai.GreatAI] class. To take advantage of it, you need to create an instance wrapping your code.
|
||||
|
||||
Let's say that you have the following greeter function:
|
||||
|
||||
|
|
@ -20,11 +20,11 @@ def greeter(your_name):
|
|||
```
|
||||
|
||||
??? info "Why not simply use `@GreatAI?`"
|
||||
The purpose of [@GreatAI.create][great_ai.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead.
|
||||
The purpose of [@GreatAI.create][great_ai.GreatAI.create] is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy. Thus, a static factory method is used instead.
|
||||
|
||||
## With types
|
||||
|
||||
Even though it's not required by GreatAI, [type annotating your codebase](https://realpython.com/python-type-checking/){ target=_blank } can save you from lots of trivial mistakes, that's why it's highly advised. Simply add the expected types to your function's signature.
|
||||
Even though it's not required by GreatAI, [type annotating your codebase](https://realpython.com/python-type-checking/){ target=_blank } can save you from lots of trivial mistakes; that's why it's highly advised. Simply add the expected types to your function's signature.
|
||||
|
||||
```python title="type_safe_greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
|
@ -34,11 +34,11 @@ def type_safe_greeter(your_name: str) -> str:
|
|||
return f'Hi {your_name}!'
|
||||
```
|
||||
|
||||
This not only allows you to statically typecheck your code, but by default, GreatAI will check it during runtime as well using [typeguard](https://github.com/agronholm/typeguard){ target=_blank }.
|
||||
This not only allows you to statically type-check your code, but by default, GreatAI will check it during runtime as well using [typeguard](https://github.com/agronholm/typeguard){ target=_blank }.
|
||||
|
||||
## With async
|
||||
|
||||
Asynchronous code can result in immense performance gains in certain cases. For example, you might rely on a third-party service, do database access, or [call a remote GreatAI instance](/how-to-guides/call-remote). In these cases, you can simply make your function `async` without any other changes.
|
||||
Asynchronous code can result in immense performance gains in some instances. For example, you might rely on a third-party service, do database access, or [call a remote GreatAI instance](/how-to-guides/call-remote). In these cases, you can make your function `async` without any other changes.
|
||||
|
||||
```python title="async_greeter.py"
|
||||
from great_ai import GreatAI
|
||||
|
|
@ -52,7 +52,7 @@ async def async_greeter(your_name: str) -> str:
|
|||
|
||||
## With decorators
|
||||
|
||||
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.GreatAI.create] must come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.
|
||||
GreatAI can decorate already decorated functions. The only restriction is that [@GreatAI.create][great_ai.GreatAI.create] must come last. There are two built-in decorators that you can use to customise your function, but you can use any third-party decorator as well.
|
||||
|
||||
### Using `@use_model`
|
||||
|
||||
|
|
@ -72,7 +72,7 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
|
|||
1. By default, the parameter named `model` will be replaced by the loaded model. This behaviour can be customised by setting the `model_kwarg_name`. This way, even multiple models can be injected into a single function.
|
||||
|
||||
!!! important
|
||||
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.GreatAI.create]. Note, that decorators are applied starting from the bottom-most one. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
You must call [@use_model][great_ai.use_model] before [@GreatAI.create][great_ai.GreatAI.create]. Note that decorators are applied starting from the bottom-most one. Feel free to use [@use_model][great_ai.use_model] in other places of the codebase, and it works equally well outside GreatAI services.
|
||||
|
||||
### Using `@parameter`
|
||||
|
||||
|
|
@ -93,11 +93,11 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
|
|||
```
|
||||
|
||||
!!! important
|
||||
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.GreatAI.create]. Note, that decorators are applied starting from the bottom-most one. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, it works equally well outside of GreatAI services.
|
||||
You must call [@parameter][great_ai.parameter] before [@GreatAI.create][great_ai.GreatAI.create]. Note that decorators are applied starting from the bottom-most one. Feel free to use [@parameter][great_ai.parameter] in other places of the codebase, and it works equally well outside GreatAI services.
|
||||
|
||||
## Complex example
|
||||
|
||||
Refer to the following example summarising the options you have when instantiating a GreatAI service.
|
||||
The following example summarises the options you have when instantiating a GreatAI service.
|
||||
|
||||
```python title="complex.py"
|
||||
from great_ai import save_model, GreatAI, parameter, use_model, log_metric
|
||||
|
|
|
|||
|
|
@ -961,7 +961,7 @@
|
|||
|
||||
|
||||
<h1 id="how-to-create-a-greatai-service">How to create a GreatAI service<a class="headerlink" href="#how-to-create-a-greatai-service" title="Permanent link">#</a></h1>
|
||||
<p>The core value of <code>great-ai</code> lies in its <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI</a> class. In order to take advantage of it, you need to create an instance wrapping your code.</p>
|
||||
<p>The core value of <code>great-ai</code> lies in its <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI</a> class. To take advantage of it, you need to create an instance wrapping your code.</p>
|
||||
<p>Let's say that you have the following greeter function:</p>
|
||||
<div class="highlight"><span class="filename">greeter.py</span><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="k">def</span> <span class="nf">my_greeter_function</span><span class="p">(</span><span class="n">your_name</span><span class="p">):</span>
|
||||
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a> <span class="k">return</span> <span class="sa">f</span><span class="s1">'Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s1">!'</span>
|
||||
|
|
@ -975,19 +975,19 @@
|
|||
</code></pre></div>
|
||||
<details class="info">
|
||||
<summary>Why not simply use <code>@GreatAI?</code></summary>
|
||||
<p>The purpose of <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for <code>__new__</code> is lacking in MyPy, thus, a static factory method is used instead.</p>
|
||||
<p>The purpose of <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for <code>__new__</code> is lacking in MyPy. Thus, a static factory method is used instead.</p>
|
||||
</details>
|
||||
<h2 id="with-types">With types<a class="headerlink" href="#with-types" title="Permanent link">#</a></h2>
|
||||
<p>Even though it's not required by GreatAI, <a href="https://realpython.com/python-type-checking/" target="_blank">type annotating your codebase</a> can save you from lots of trivial mistakes, that's why it's highly advised. Simply add the expected types to your function's signature.</p>
|
||||
<p>Even though it's not required by GreatAI, <a href="https://realpython.com/python-type-checking/" target="_blank">type annotating your codebase</a> can save you from lots of trivial mistakes; that's why it's highly advised. Simply add the expected types to your function's signature.</p>
|
||||
<div class="highlight"><span class="filename">type_safe_greeter.py</span><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span>
|
||||
<a id="__codelineno-2-2" name="__codelineno-2-2" href="#__codelineno-2-2"></a>
|
||||
<a id="__codelineno-2-3" name="__codelineno-2-3" href="#__codelineno-2-3"></a><span class="nd">@GreatAI</span><span class="o">.</span><span class="n">create</span>
|
||||
<a id="__codelineno-2-4" name="__codelineno-2-4" href="#__codelineno-2-4"></a><span class="k">def</span> <span class="nf">type_safe_greeter</span><span class="p">(</span><span class="n">your_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
<a id="__codelineno-2-5" name="__codelineno-2-5" href="#__codelineno-2-5"></a> <span class="k">return</span> <span class="sa">f</span><span class="s1">'Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s1">!'</span>
|
||||
</code></pre></div>
|
||||
<p>This not only allows you to statically typecheck your code, but by default, GreatAI will check it during runtime as well using <a href="https://github.com/agronholm/typeguard" target="_blank">typeguard</a>.</p>
|
||||
<p>This not only allows you to statically type-check your code, but by default, GreatAI will check it during runtime as well using <a href="https://github.com/agronholm/typeguard" target="_blank">typeguard</a>.</p>
|
||||
<h2 id="with-async">With async<a class="headerlink" href="#with-async" title="Permanent link">#</a></h2>
|
||||
<p>Asynchronous code can result in immense performance gains in certain cases. For example, you might rely on a third-party service, do database access, or <a href="/how-to-guides/call-remote">call a remote GreatAI instance</a>. In these cases, you can simply make your function <code>async</code> without any other changes.</p>
|
||||
<p>Asynchronous code can result in immense performance gains in some instances. For example, you might rely on a third-party service, do database access, or <a href="/how-to-guides/call-remote">call a remote GreatAI instance</a>. In these cases, you can make your function <code>async</code> without any other changes.</p>
|
||||
<div class="highlight"><span class="filename">async_greeter.py</span><pre><span></span><code><a id="__codelineno-3-1" name="__codelineno-3-1" href="#__codelineno-3-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span>
|
||||
<a id="__codelineno-3-2" name="__codelineno-3-2" href="#__codelineno-3-2"></a><span class="kn">from</span> <span class="nn">asyncio</span> <span class="kn">import</span> <span class="n">sleep</span>
|
||||
<a id="__codelineno-3-3" name="__codelineno-3-3" href="#__codelineno-3-3"></a>
|
||||
|
|
@ -997,7 +997,7 @@
|
|||
<a id="__codelineno-3-7" name="__codelineno-3-7" href="#__codelineno-3-7"></a> <span class="k">return</span> <span class="sa">f</span><span class="s1">'Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s1">!'</span>
|
||||
</code></pre></div>
|
||||
<h2 id="with-decorators">With decorators<a class="headerlink" href="#with-decorators" title="Permanent link">#</a></h2>
|
||||
<p>GreatAI can decorate already decorated functions. The only restriction is that <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> must come last. There are two built-in decorators that you can use to customise your function but you can you use any third-party decorator as well.</p>
|
||||
<p>GreatAI can decorate already decorated functions. The only restriction is that <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> must come last. There are two built-in decorators that you can use to customise your function, but you can use any third-party decorator as well.</p>
|
||||
<h3 id="using-use_model">Using <code>@use_model</code><a class="headerlink" href="#using-use_model" title="Permanent link">#</a></h3>
|
||||
<p>If you have previously saved a model with <a class="autorefs autorefs-internal" href="../../reference/#great_ai.save_model">save_model</a>, you can inject it into your function by calling <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a>.</p>
|
||||
<div class="highlight"><span class="filename">greeter_with_model.py</span><pre><span></span><code><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span><span class="p">,</span> <span class="n">use_model</span>
|
||||
|
|
@ -1014,7 +1014,7 @@
|
|||
</ol>
|
||||
<div class="admonition important">
|
||||
<p class="admonition-title">Important</p>
|
||||
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Note, that decorators are applied starting from the bottom-most one. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> in other places of the codebase, it works equally well outside of GreatAI services. </p>
|
||||
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Note that decorators are applied starting from the bottom-most one. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> in other places of the codebase, and it works equally well outside GreatAI services. </p>
|
||||
</div>
|
||||
<h3 id="using-parameter">Using <code>@parameter</code><a class="headerlink" href="#using-parameter" title="Permanent link">#</a></h3>
|
||||
<p>If you wish to turn off logging or specify custom validation for your parameters, you can use the <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> decorator.</p>
|
||||
|
|
@ -1033,10 +1033,10 @@
|
|||
</code></pre></div>
|
||||
<div class="admonition important">
|
||||
<p class="admonition-title">Important</p>
|
||||
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Note, that decorators are applied starting from the bottom-most one. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> in other places of the codebase, it works equally well outside of GreatAI services. </p>
|
||||
<p>You must call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> before <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a>. Note that decorators are applied starting from the bottom-most one. Feel free to use <a class="autorefs autorefs-internal" href="../../reference/#great_ai.parameter">@parameter</a> in other places of the codebase, and it works equally well outside GreatAI services. </p>
|
||||
</div>
|
||||
<h2 id="complex-example">Complex example<a class="headerlink" href="#complex-example" title="Permanent link">#</a></h2>
|
||||
<p>Refer to the following example summarising the options you have when instantiating a GreatAI service.</p>
|
||||
<p>The following example summarises the options you have when instantiating a GreatAI service.</p>
|
||||
<div class="highlight"><span class="filename">complex.py</span><pre><span></span><code><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">save_model</span><span class="p">,</span> <span class="n">GreatAI</span><span class="p">,</span> <span class="n">parameter</span><span class="p">,</span> <span class="n">use_model</span><span class="p">,</span> <span class="n">log_metric</span>
|
||||
<a id="__codelineno-6-2" name="__codelineno-6-2" href="#__codelineno-6-2"></a>
|
||||
<a id="__codelineno-6-3" name="__codelineno-6-3" href="#__codelineno-6-3"></a><span class="n">save_model</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="s1">'secret-number'</span><span class="p">)</span> <span class="c1">#(1)</span>
|
||||
|
|
@ -1062,7 +1062,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -1,13 +1,13 @@
|
|||
# How to manage training data
|
||||
|
||||
In order to simplify your training data management, `great-ai` provide two complementing approaches for inputting new data-points.
|
||||
In order to simplify your training data management, `great-ai` provide two complementing approaches for inputting new data points.
|
||||
|
||||
## Upload data
|
||||
|
||||
At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call [great_ai.add_ground_truth][]. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, [great_ai.query_ground_truth][] can be called to get a filtered view of the training data.
|
||||
|
||||
!!! important "Train-test-validation splits"
|
||||
It is a best practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.
|
||||
It is a best practice to lock away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results; hence, care must be taken to avoid it.
|
||||
|
||||
With [great_ai.add_ground_truth][], there is an option to tag the samples with `train`, `test`, and `validation` randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.
|
||||
|
||||
|
|
@ -50,13 +50,13 @@ add_ground_truth(
|
|||
'trace_id': 'abee0671-beb9-4284-8c3b-c65e5836ce38'})]
|
||||
```
|
||||
|
||||
1. Expected output. This can be also accessed through the `.output` property.
|
||||
1. Expected output. This can also be accessed through the `.output` property.
|
||||
2. The input value is stored here.
|
||||
3. Notice how `ground_truth` is always included as a tag when using [great_ai.add_ground_truth][].
|
||||
|
||||
## Get feedback
|
||||
|
||||
After the initial data gathering, end-to-end feedback can be also integrated into the dataset.
|
||||
After the initial data gathering, end-to-end feedback can also be integrated into the dataset.
|
||||
|
||||
The scaffolded REST API contains endpoints for managing traces and their feedbacks.
|
||||
|
||||
|
|
|
|||
|
|
@ -907,12 +907,12 @@
|
|||
|
||||
|
||||
<h1 id="how-to-manage-training-data">How to manage training data<a class="headerlink" href="#how-to-manage-training-data" title="Permanent link">#</a></h1>
|
||||
<p>In order to simplify your training data management, <code>great-ai</code> provide two complementing approaches for inputting new data-points.</p>
|
||||
<p>In order to simplify your training data management, <code>great-ai</code> provide two complementing approaches for inputting new data points.</p>
|
||||
<h2 id="upload-data">Upload data<a class="headerlink" href="#upload-data" title="Permanent link">#</a></h2>
|
||||
<p>At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, <a class="autorefs autorefs-internal" href="../../reference/#great_ai.query_ground_truth">great_ai.query_ground_truth</a> can be called to get a filtered view of the training data.</p>
|
||||
<div class="admonition important">
|
||||
<p class="admonition-title">Train-test-validation splits</p>
|
||||
<p>It is a best practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.</p>
|
||||
<p>It is a best practice to lock away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results; hence, care must be taken to avoid it.</p>
|
||||
<p>With <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>, there is an option to tag the samples with <code>train</code>, <code>test</code>, and <code>validation</code> randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.</p>
|
||||
</div>
|
||||
<p>The nice thing about this is that the 'input-expected output' pairs are stored as traces. Thus, they behave exactly like regular prediction traces.</p>
|
||||
|
|
@ -951,12 +951,12 @@
|
|||
<a id="__codelineno-1-20" name="__codelineno-1-20" href="#__codelineno-1-20"></a> <span class="s1">'trace_id'</span><span class="p">:</span> <span class="s1">'abee0671-beb9-4284-8c3b-c65e5836ce38'</span><span class="p">})]</span>
|
||||
</code></pre></div>
|
||||
<ol>
|
||||
<li>Expected output. This can be also accessed through the <code>.output</code> property.</li>
|
||||
<li>Expected output. This can also be accessed through the <code>.output</code> property.</li>
|
||||
<li>The input value is stored here.</li>
|
||||
<li>Notice how <code>ground_truth</code> is always included as a tag when using <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>. </li>
|
||||
</ol>
|
||||
<h2 id="get-feedback">Get feedback<a class="headerlink" href="#get-feedback" title="Permanent link">#</a></h2>
|
||||
<p>After the initial data gathering, end-to-end feedback can be also integrated into the dataset. </p>
|
||||
<p>After the initial data gathering, end-to-end feedback can also be integrated into the dataset. </p>
|
||||
<p>The scaffolded REST API contains endpoints for managing traces and their feedbacks.</p>
|
||||
<p><img alt="screenshot of swagger" loading="lazy" src="/media/feedback.png" /></p>
|
||||
<p>When <a class="autorefs autorefs-internal" href="../../reference/#great_ai.query_ground_truth">great_ai.query_ground_truth</a> is executed, it implicitly filters for traces that have feedback. Therefore, both the <code>ground_truth</code> and the <code>online</code> traces that have received feedback are returned. No matter the origin of the data, it can be accessed using the same API.</p>
|
||||
|
|
@ -968,7 +968,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -936,23 +936,23 @@
|
|||
<p>This will make GreatAI work in Colab.</p>
|
||||
</blockquote>
|
||||
<h2 id="command-line-tools">Command-line tools<a class="headerlink" href="#command-line-tools" title="Permanent link">#</a></h2>
|
||||
<p>After installation, <code>great-ai</code> and <code>large-file</code> are available as commands. The former is required for deploying your application while the latter lets you manage models and datasets from your terminal.</p>
|
||||
<p>After installation, <code>great-ai</code> and <code>large-file</code> are available as commands. The former is required for deploying your application, while the latter lets you manage models and datasets from your terminal.</p>
|
||||
<details class="note">
|
||||
<summary>Snakes & kebabs</summary>
|
||||
<p>The library is called <code>great-ai</code>, therefore, its command-line entrypoint is also called <code>great-ai</code>. However, Python module names cannot contain hyphens, that's why you have to <code>import great_ai</code> with an underscore. The <code>great-ai</code> CLI tool is also available as <code>python3 -m great_ai</code>.</p>
|
||||
<p>To help with the confusion, a CLI executable called <code>great_ai</code> (and <code>large_file</code>) are also installed, thus, if you prefer, you can always refer to GreatAI using its underscored name variant (<code>great_ai</code>).</p>
|
||||
<p>The library is called <code>great-ai</code>; therefore, its command-line entry point is also called <code>great-ai</code>. However, Python module names cannot contain hyphens, that's why you have to <code>import great_ai</code> with an underscore. The <code>great-ai</code> CLI tool is also available as <code>python3 -m great_ai</code>.</p>
|
||||
<p>To help with the confusion, a CLI executable called <code>great_ai</code> (and <code>large_file</code>) are also installed. Thus, if you prefer, you can always refer to GreatAI using its underscored name variant (<code>great_ai</code>).</p>
|
||||
</details>
|
||||
<div class="admonition warning">
|
||||
<p class="admonition-title">Windows</p>
|
||||
<p>On Windows, you might encounter a similar warning from <code>pip</code>:</p>
|
||||
<blockquote>
|
||||
<p><code>WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users\...\Scripts' which is not on PATH.</code></p>
|
||||
<p><code>WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users\...\Scripts', which is not on PATH.</code></p>
|
||||
<p><code>Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.</code></p>
|
||||
</blockquote>
|
||||
<p>This means that <code>great-ai.exe</code> and <code>large-file.exe</code> are not in your <code>PATH</code>. Either add their containing directory ('C:\Users...\Scripts' in this case) to your <code>PATH</code> or use <code>python3 -m great_ai</code> and <code>python3 -m great_ai.large_file</code> instead of the exe-s.</p>
|
||||
</div>
|
||||
<h2 id="update">Update<a class="headerlink" href="#update" title="Permanent link">#</a></h2>
|
||||
<p>If you wish to update to the latest version execute:</p>
|
||||
<p>If you wish to update to the latest version, execute:</p>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a>pip install --upgrade great-ai
|
||||
</code></pre></div>
|
||||
<h2 id="bleeding-edge">Bleeding edge<a class="headerlink" href="#bleeding-edge" title="Permanent link">#</a></h2>
|
||||
|
|
@ -968,7 +968,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 29, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -20,16 +20,16 @@ pip install great-ai pyyaml==5.4.1
|
|||
|
||||
## Command-line tools
|
||||
|
||||
After installation, `great-ai` and `large-file` are available as commands. The former is required for deploying your application while the latter lets you manage models and datasets from your terminal.
|
||||
After installation, `great-ai` and `large-file` are available as commands. The former is required for deploying your application, while the latter lets you manage models and datasets from your terminal.
|
||||
|
||||
??? note "Snakes & kebabs"
|
||||
The library is called `great-ai`, therefore, its command-line entrypoint is also called `great-ai`. However, Python module names cannot contain hyphens, that's why you have to `import great_ai` with an underscore. The `great-ai` CLI tool is also available as `python3 -m great_ai`.
|
||||
The library is called `great-ai`; therefore, its command-line entry point is also called `great-ai`. However, Python module names cannot contain hyphens, that's why you have to `import great_ai` with an underscore. The `great-ai` CLI tool is also available as `python3 -m great_ai`.
|
||||
|
||||
To help with the confusion, a CLI executable called `great_ai` (and `large_file`) are also installed, thus, if you prefer, you can always refer to GreatAI using its underscored name variant (`great_ai`).
|
||||
To help with the confusion, a CLI executable called `great_ai` (and `large_file`) are also installed. Thus, if you prefer, you can always refer to GreatAI using its underscored name variant (`great_ai`).
|
||||
|
||||
!!! warning "Windows"
|
||||
On Windows, you might encounter a similar warning from `pip`:
|
||||
> `WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users\...\Scripts' which is not on PATH.`
|
||||
> `WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users\...\Scripts', which is not on PATH.`
|
||||
|
||||
> `Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.`
|
||||
|
||||
|
|
@ -37,7 +37,7 @@ After installation, `great-ai` and `large-file` are available as commands. The f
|
|||
|
||||
## Update
|
||||
|
||||
If you wish to update to the latest version execute:
|
||||
If you wish to update to the latest version, execute:
|
||||
|
||||
```sh
|
||||
pip install --upgrade great-ai
|
||||
|
|
|
|||
|
|
@ -1029,11 +1029,11 @@
|
|||
<h1 id="how-to-use-largefiles">How to use LargeFiles<a class="headerlink" href="#how-to-use-largefiles" title="Permanent link">#</a></h1>
|
||||
<p>The functions <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">save_model</a> and <a class="autorefs autorefs-internal" href="../../reference/#great_ai.use_model">@use_model</a> wrap LargeFile instances. Hence, besides configuring <a href="/reference/large-file">LargeFile</a>, users have few reasons to use LargeFiles directly.</p>
|
||||
<h2 id="motivation">Motivation<a class="headerlink" href="#motivation" title="Permanent link">#</a></h2>
|
||||
<p>Oftentimes, especially when working with data-heavy applications, large files can proliferate in a repository. Version controlling them is an obvious next step, however, GitHub's git LFS implementation <a href="https://docs.github.com/en/repositories/working-with-files/managing-large-files/removing-files-from-git-large-file-storage#git-lfs-objects-in-your-repository">doesn't support deleting</a> large files, making it easy for them to eat-up the LFS quota and explode the size of your repos.</p>
|
||||
<p><a href="https://dvc.org/">DVC</a> is a viable alternative, however, it requires users to learn to use one more CLI tool.</p>
|
||||
<p>Often, especially when working with data-heavy applications, large files can proliferate in a repository. Version controlling them is an obvious next step. However, GitHub's git LFS implementation <a href="https://docs.github.com/en/repositories/working-with-files/managing-large-files/removing-files-from-git-large-file-storage#git-lfs-objects-in-your-repository">doesn't support deleting</a> large files, making it easy for them to eat-up the LFS quota and explode the size of your repos.</p>
|
||||
<p><a href="https://dvc.org/">DVC</a> is a viable alternative; however, it requires users to learn to use one more CLI tool.</p>
|
||||
<details class="note">
|
||||
<summary>Using LargeFile-s directly (usually not needed)</summary>
|
||||
<p>LargeFile doesn't require users to learn too much new. It is a nearly exact copy of the built-in <code>open()</code> function of Python with which users are certainly already familiar.</p>
|
||||
<p>LargeFile doesn't require users to learn too much new. It is a nearly exact copy of Python's built-in <code>open()</code> function, with which users are undoubtedly already familiar.</p>
|
||||
<h2 id="simple-example">Simple example<a class="headerlink" href="#simple-example" title="Permanent link">#</a></h2>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="kn">from</span> <span class="nn">great_ai.large_file</span> <span class="kn">import</span> <span class="n">LargeFileS3</span>
|
||||
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a>
|
||||
|
|
@ -1045,7 +1045,7 @@
|
|||
<a id="__codelineno-0-8" name="__codelineno-0-8" href="#__codelineno-0-8"></a><span class="p">})</span>
|
||||
<a id="__codelineno-0-9" name="__codelineno-0-9" href="#__codelineno-0-9"></a>
|
||||
<a id="__codelineno-0-10" name="__codelineno-0-10" href="#__codelineno-0-10"></a><span class="c1"># Creates a new version and deletes the older version </span>
|
||||
<a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a><span class="c1"># leaving the 3 most recently used intact</span>
|
||||
<a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a><span class="c1"># leaving the three most recently used intact</span>
|
||||
<a id="__codelineno-0-12" name="__codelineno-0-12" href="#__codelineno-0-12"></a><span class="k">with</span> <span class="n">LargeFileS3</span><span class="p">(</span><span class="s2">"test.txt"</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">,</span> <span class="n">keep_last_n</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-13" name="__codelineno-0-13" href="#__codelineno-0-13"></a> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100000</span><span class="p">):</span>
|
||||
<a id="__codelineno-0-14" name="__codelineno-0-14" href="#__codelineno-0-14"></a> <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">'test</span><span class="se">\n</span><span class="s1">'</span><span class="p">)</span>
|
||||
|
|
@ -1056,10 +1056,10 @@
|
|||
<a id="__codelineno-0-19" name="__codelineno-0-19" href="#__codelineno-0-19"></a> <span class="nb">print</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">readlines</span><span class="p">()[</span><span class="mi">0</span><span class="p">])</span>
|
||||
</code></pre></div>
|
||||
<ol>
|
||||
<li>In this case, the latest version is already in the local cache, no download is required.</li>
|
||||
<li>The latest version is already in the local cache; no download is required.</li>
|
||||
</ol>
|
||||
<h3 id="more-details">More details<a class="headerlink" href="#more-details" title="Permanent link">#</a></h3>
|
||||
<p><code>LargeFile</code> behaves like an opened file (in the background it is a temp file after all). Binary reads and writes are supported along with the <a href="https://docs.python.org/3/library/functions.html#open" target="_blank">different keywords <code>open()</code> accepts</a>.</p>
|
||||
<p><code>LargeFile</code> behaves like an opened file (in the background, it is a temp file after all). Binary reads and writes are supported along with the <a href="https://docs.python.org/3/library/functions.html#open" target="_blank">different keywords <code>open()</code> accepts</a>.</p>
|
||||
<p>The local cache can be configured with these properties:</p>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a><span class="n">LargeFileS3</span><span class="o">.</span><span class="n">cache_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s1">'.cache'</span><span class="p">)</span>
|
||||
<a id="__codelineno-1-2" name="__codelineno-1-2" href="#__codelineno-1-2"></a><span class="n">LargeFileS3</span><span class="o">.</span><span class="n">max_cache_size</span> <span class="o">=</span> <span class="s2">"30 GB"</span>
|
||||
|
|
@ -1075,7 +1075,7 @@
|
|||
<div class="highlight"><pre><span></span><code><a id="__codelineno-3-1" name="__codelineno-3-1" href="#__codelineno-3-1"></a><span class="n">LargeFileS3</span><span class="p">(</span><span class="s2">"folder-of-my-bert-model"</span><span class="p">)</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="s1">'path_to_local/folder_or_file'</span><span class="p">)</span>
|
||||
</code></pre></div>
|
||||
<blockquote>
|
||||
<p>This way both regular files and folders can be handled. The uploaded file is called <strong>folder-of-my-bert-model</strong>, the local name is ignored.</p>
|
||||
<p>This way, both regular files and folders can be handled. The uploaded file is called <strong>folder-of-my-bert-model</strong>, the local name is ignored.</p>
|
||||
</blockquote>
|
||||
<p>Lastly, all version of the remote object can be deleted by calling <code>LargeFileS3("my-file").delete()</code>. It will still reside in your local cache afterwards; its deletion will happen next time your local cache has to be pruned.</p>
|
||||
</details>
|
||||
|
|
@ -1093,14 +1093,14 @@
|
|||
<a id="__codelineno-5-2" name="__codelineno-5-2" href="#__codelineno-5-2"></a> --push my_first_file.json folder/my_second_file my_folder
|
||||
</code></pre></div>
|
||||
<blockquote>
|
||||
<p>Only the filename is used as the S3 name, the rest of the path is ignored.</p>
|
||||
<p>Only the filename is used as the S3 name; the rest of the path is ignored.</p>
|
||||
</blockquote>
|
||||
<div class="admonition important">
|
||||
<p class="admonition-title">Using MongoDB</p>
|
||||
<p>The possible values for <code>--backend</code> are <code>s3</code>, <code>mongo</code>, and <code>local</code>. The latter doesn't need credentials, it only versions and stores your files in a local folder. MongoDB on the other hand requires a <code>mongo_connection_string</code> and a <code>mongo_database</code> to be specified. For storing large files, it uses the <a href="https://www.mongodb.com/docs/manual/core/gridfs" target="_blank">GridFS</a> specification.</p>
|
||||
<p>The possible values for <code>--backend</code> are <code>s3</code>, <code>mongo</code>, and <code>local</code>. The latter doesn't need credentials. It only versions and stores your files in a local folder. MongoDB, on the other hand, requires a <code>mongo_connection_string</code> and a <code>mongo_database</code> to be specified. For storing large files, it uses the <a href="https://www.mongodb.com/docs/manual/core/gridfs" target="_blank">GridFS</a> specification.</p>
|
||||
</div>
|
||||
<h3 id="download-some-files-to-the-local-cache">Download some files to the local cache<a class="headerlink" href="#download-some-files-to-the-local-cache" title="Permanent link">#</a></h3>
|
||||
<p>This can be useful when building a Docker image for example. This way, the files can already reside inside the container and need not be downloaded later.</p>
|
||||
<p>This can be useful when building a Docker image, for example. This way, the files can already reside inside the container and need not be downloaded later.</p>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a>large-file --backend s3 --secrets ~/.aws/credentials <span class="se">\</span>
|
||||
<a id="__codelineno-6-2" name="__codelineno-6-2" href="#__codelineno-6-2"></a> --cache my_first_file.json:3 my_second_file my_folder:0
|
||||
</code></pre></div>
|
||||
|
|
@ -1117,7 +1117,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -4,12 +4,12 @@ The functions [save_model][great_ai.use_model] and [@use_model][great_ai.use_mod
|
|||
|
||||
## Motivation
|
||||
|
||||
Oftentimes, especially when working with data-heavy applications, large files can proliferate in a repository. Version controlling them is an obvious next step, however, GitHub's git LFS implementation [doesn't support deleting](https://docs.github.com/en/repositories/working-with-files/managing-large-files/removing-files-from-git-large-file-storage#git-lfs-objects-in-your-repository) large files, making it easy for them to eat-up the LFS quota and explode the size of your repos.
|
||||
Often, especially when working with data-heavy applications, large files can proliferate in a repository. Version controlling them is an obvious next step. However, GitHub's git LFS implementation [doesn't support deleting](https://docs.github.com/en/repositories/working-with-files/managing-large-files/removing-files-from-git-large-file-storage#git-lfs-objects-in-your-repository) large files, making it easy for them to eat-up the LFS quota and explode the size of your repos.
|
||||
|
||||
[DVC](https://dvc.org/) is a viable alternative, however, it requires users to learn to use one more CLI tool.
|
||||
[DVC](https://dvc.org/) is a viable alternative; however, it requires users to learn to use one more CLI tool.
|
||||
|
||||
??? note "Using LargeFile-s directly (usually not needed)"
|
||||
LargeFile doesn't require users to learn too much new. It is a nearly exact copy of the built-in `open()` function of Python with which users are certainly already familiar.
|
||||
LargeFile doesn't require users to learn too much new. It is a nearly exact copy of Python's built-in `open()` function, with which users are undoubtedly already familiar.
|
||||
|
||||
## Simple example
|
||||
|
||||
|
|
@ -24,7 +24,7 @@ Oftentimes, especially when working with data-heavy applications, large files ca
|
|||
})
|
||||
|
||||
# Creates a new version and deletes the older version
|
||||
# leaving the 3 most recently used intact
|
||||
# leaving the three most recently used intact
|
||||
with LargeFileS3("test.txt", "w", keep_last_n=3) as f:
|
||||
for i in range(100000):
|
||||
f.write('test\n')
|
||||
|
|
@ -35,11 +35,11 @@ Oftentimes, especially when working with data-heavy applications, large files ca
|
|||
print(f.readlines()[0])
|
||||
```
|
||||
|
||||
1. In this case, the latest version is already in the local cache, no download is required.
|
||||
1. The latest version is already in the local cache; no download is required.
|
||||
|
||||
### More details
|
||||
|
||||
`LargeFile` behaves like an opened file (in the background it is a temp file after all). Binary reads and writes are supported along with the [different keywords `open()` accepts](https://docs.python.org/3/library/functions.html#open){ target=_blank }.
|
||||
`LargeFile` behaves like an opened file (in the background, it is a temp file after all). Binary reads and writes are supported along with the [different keywords `open()` accepts](https://docs.python.org/3/library/functions.html#open){ target=_blank }.
|
||||
|
||||
The local cache can be configured with these properties:
|
||||
|
||||
|
|
@ -64,7 +64,7 @@ Oftentimes, especially when working with data-heavy applications, large files ca
|
|||
LargeFileS3("folder-of-my-bert-model").push('path_to_local/folder_or_file')
|
||||
```
|
||||
|
||||
> This way both regular files and folders can be handled. The uploaded file is called **folder-of-my-bert-model**, the local name is ignored.
|
||||
> This way, both regular files and folders can be handled. The uploaded file is called **folder-of-my-bert-model**, the local name is ignored.
|
||||
|
||||
Lastly, all version of the remote object can be deleted by calling `LargeFileS3("my-file").delete()`. It will still reside in your local cache afterwards; its deletion will happen next time your local cache has to be pruned.
|
||||
|
||||
|
|
@ -90,14 +90,14 @@ large-file --backend s3 --secrets secrets.ini \
|
|||
--push my_first_file.json folder/my_second_file my_folder
|
||||
```
|
||||
|
||||
> Only the filename is used as the S3 name, the rest of the path is ignored.
|
||||
> Only the filename is used as the S3 name; the rest of the path is ignored.
|
||||
|
||||
!!! important "Using MongoDB"
|
||||
The possible values for `--backend` are `s3`, `mongo`, and `local`. The latter doesn't need credentials, it only versions and stores your files in a local folder. MongoDB on the other hand requires a `mongo_connection_string` and a `mongo_database` to be specified. For storing large files, it uses the [GridFS](https://www.mongodb.com/docs/manual/core/gridfs){ target=_blank } specification.
|
||||
The possible values for `--backend` are `s3`, `mongo`, and `local`. The latter doesn't need credentials. It only versions and stores your files in a local folder. MongoDB, on the other hand, requires a `mongo_connection_string` and a `mongo_database` to be specified. For storing large files, it uses the [GridFS](https://www.mongodb.com/docs/manual/core/gridfs){ target=_blank } specification.
|
||||
|
||||
### Download some files to the local cache
|
||||
|
||||
This can be useful when building a Docker image for example. This way, the files can already reside inside the container and need not be downloaded later.
|
||||
This can be useful when building a Docker image, for example. This way, the files can already reside inside the container and need not be downloaded later.
|
||||
|
||||
```sh
|
||||
large-file --backend s3 --secrets ~/.aws/credentials \
|
||||
|
|
|
|||
|
|
@ -1025,7 +1025,7 @@
|
|||
<p>As you can see, the original return value is wrapped in a <a class="autorefs autorefs-internal" href="../../reference/views/#great_ai.Trace">Trace</a> object (which is also persisted in your database of choice). You can access the original value under the <code>output</code> property.</p>
|
||||
<h2 id="online-prediction">Online prediction<a class="headerlink" href="#online-prediction" title="Permanent link">#</a></h2>
|
||||
<p>Likely, the main way you would like to expose your model is through an HTTP API. <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> scaffolds many REST API endpoints for your model and creates a <a href="https://fastapi.tiangolo.com/" target="_blank">FastAPI</a> app available under <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI.app</a>. This can be served using <a href="https://www.uvicorn.org/" target="_blank">uvicorn</a> or any other <a href="https://asgi.readthedocs.io/en/latest/" target="_blank">ASGI server</a>.</p>
|
||||
<p>Since most ML code lives in <a href="https://jupyter.org/" target="_blank">Jupyter</a> notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, <code>uvicorn</code> is wrapped by the <code>great-ai</code> command-line utility which — among others — takes care of feeding a notebook into <code>uvicorn</code>. It also supports auto-reloading.</p>
|
||||
<p>Since most ML code lives in <a href="https://jupyter.org/" target="_blank">Jupyter</a> notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, <code>uvicorn</code> is wrapped by the <code>great-ai</code> command-line utility, which — among others — takes care of feeding a notebook into <code>uvicorn</code>. It also supports auto-reloading.</p>
|
||||
<h3 id="in-development">In development<a class="headerlink" href="#in-development" title="Permanent link">#</a></h3>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a>great-ai greeter.py
|
||||
</code></pre></div>
|
||||
|
|
@ -1072,10 +1072,10 @@
|
|||
<a id="__codelineno-6-9" name="__codelineno-6-9" href="#__codelineno-6-9"></a><span class="c"># have to download it each time a container starts</span>
|
||||
<a id="__codelineno-6-10" name="__codelineno-6-10" href="#__codelineno-6-10"></a><span class="k">RUN</span><span class="w"> </span>large-file --backend s3 --secrets s3.ini --cache my-domain-predictor
|
||||
<a id="__codelineno-6-11" name="__codelineno-6-11" href="#__codelineno-6-11"></a>
|
||||
<a id="__codelineno-6-12" name="__codelineno-6-12" href="#__codelineno-6-12"></a><span class="c"># Add you application code to the image</span>
|
||||
<a id="__codelineno-6-12" name="__codelineno-6-12" href="#__codelineno-6-12"></a><span class="c"># Add your application code to the image</span>
|
||||
<a id="__codelineno-6-13" name="__codelineno-6-13" href="#__codelineno-6-13"></a><span class="k">COPY</span><span class="w"> </span>. .
|
||||
<a id="__codelineno-6-14" name="__codelineno-6-14" href="#__codelineno-6-14"></a>
|
||||
<a id="__codelineno-6-15" name="__codelineno-6-15" href="#__codelineno-6-15"></a><span class="c"># The default ENTRYPOINT is great-ai, specify its argument using CMD</span>
|
||||
<a id="__codelineno-6-15" name="__codelineno-6-15" href="#__codelineno-6-15"></a><span class="c"># The default ENTRYPOINT is great-ai; specify its argument using CMD</span>
|
||||
<a id="__codelineno-6-16" name="__codelineno-6-16" href="#__codelineno-6-16"></a><span class="k">CMD</span><span class="w"> </span><span class="p">[</span><span class="s2">"deploy.ipynb"</span><span class="p">]</span>
|
||||
</code></pre></div>
|
||||
<h2 id="batch-prediction">Batch prediction<a class="headerlink" href="#batch-prediction" title="Permanent link">#</a></h2>
|
||||
|
|
@ -1106,7 +1106,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ As you can see, the original return value is wrapped in a [Trace][great_ai.Trace
|
|||
|
||||
Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
|
||||
|
||||
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, `uvicorn` is wrapped by the `great-ai` command-line utility which — among others — takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
|
||||
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, `uvicorn` is wrapped by the `great-ai` command-line utility, which — among others — takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
|
||||
|
||||
### In development
|
||||
|
||||
|
|
@ -97,12 +97,11 @@ RUN pip install --no-cache-dir --requirement requirements.txt
|
|||
# have to download it each time a container starts
|
||||
RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor
|
||||
|
||||
# Add you application code to the image
|
||||
# Add your application code to the image
|
||||
COPY . .
|
||||
|
||||
# The default ENTRYPOINT is great-ai, specify its argument using CMD
|
||||
# The default ENTRYPOINT is great-ai; specify its argument using CMD
|
||||
CMD ["deploy.ipynb"]
|
||||
|
||||
```
|
||||
|
||||
## Batch prediction
|
||||
|
|
|
|||
16
index.html
16
index.html
|
|
@ -951,18 +951,18 @@
|
|||
<a href="https://github.com/schmelczer/great-ai/actions/workflows/test.yml"><img alt="Test" src="https://github.com/schmelczer/great-ai/actions/workflows/test.yml/badge.svg" /></a>
|
||||
<a href="https://sonar.scoutinscience.com/dashboard?id=great-ai"><img alt="Sonar line coverage" src="https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=coverage" /></a>
|
||||
<a href="https://sonar.scoutinscience.com/dashboard?id=great-ai"><img alt="Sonar LoC" src="https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=ncloc" /></a></p>
|
||||
<p>Applying AI is becoming increasingly easier but many case studies have shown that these applications are often deployed poorly. This may lead to suboptimal performance and to introducing <a href="https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction" target="_blank">unintended biases</a>. GreatAI helps fixing this by allowing you to <mark>easily transform your prototype AI code into production-ready software</mark>.</p>
|
||||
<p>Applying AI is becoming increasingly easier, but many case studies have shown that these applications are often deployed poorly. This may lead to suboptimal performance and to introducing <a href="https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction" target="_blank">unintended biases</a>. GreatAI helps fix this by allowing you to <mark>easily transform your prototype AI code into production-ready software</mark>.</p>
|
||||
<details class="quote">
|
||||
<summary>Case studies</summary>
|
||||
<p>"There is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature."
|
||||
— <a href="https://ieeexplore.ieee.org/abstract/document/9359253" target="_blank">John et al.</a></p>
|
||||
<p>"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." — <a href="https://link.springer.com/article/10.1007/s10664-021-09993-1" target="_blank">Haakman et al.</a></p>
|
||||
<p>"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." — <a href="https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf" target="_blank">Sculley et al.</a></p>
|
||||
<p>"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. — <a href="https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw" target="_blank">Serban et al.</a></p>
|
||||
<p>"For example, practice 25 is very important for "Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. — <a href="https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw" target="_blank">Serban et al.</a></p>
|
||||
</details>
|
||||
<h2 id="features">Features<a class="headerlink" href="#features" title="Permanent link">#</a></h2>
|
||||
<ul class="task-list">
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save prediction traces of each prediction including arguments and model versions</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save prediction traces of each prediction, including arguments and model versions</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save feedback and merge it into a ground-truth database</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Version and store models and data on shared infrastructure <em>(MongoDB GridFS, S3-compatible storage, shared volume)</em></li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration</li>
|
||||
|
|
@ -1012,7 +1012,7 @@
|
|||
</ol>
|
||||
<details class="note">
|
||||
<summary>Note</summary>
|
||||
<p>In practice, <code>greeter</code> could be an inference function of some AI/ML application. But it could also just wrap a black-box solution of some SaaS. Either way, it is <a href="https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai" target="_blank">imperative to have continuous oversight</a> of the services you provide and data you process especially in the context of AI/ML applications.</p>
|
||||
<p>In practice, <code>greeter</code> could be an inference function of some AI/ML application. But it could also just wrap a black-box solution of some SaaS. Either way, it is <a href="https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai" target="_blank">imperative to have continuous oversight</a> of the services you provide and the data you process, especially in the context of AI/ML applications.</p>
|
||||
</details>
|
||||
<div class="highlight"><span class="filename">terminal</span><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a>great-ai demo.py
|
||||
</code></pre></div>
|
||||
|
|
@ -1026,7 +1026,7 @@
|
|||
</div>
|
||||
<h2 id="why-is-this-great">Why is this GREAT?<a class="headerlink" href="#why-is-this-great" title="Permanent link">#</a></h2>
|
||||
<p><img alt="scope of GreatAI" src="media/scope-simple.drawio.svg" /></p>
|
||||
<p>GreatAI fits between the prototype and deployment phases of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, several best practices can be automatically implemented aiming to achieve the following attributes:</p>
|
||||
<p>GreatAI fits between the prototype and deployment phases of your (or your organisation's) AI development lifecycle. This is highlighted in blue in the diagram. Here, several best practices can be automatically implemented, aiming to achieve the following attributes:</p>
|
||||
<ul>
|
||||
<li><strong>G</strong>eneral: use any Python library without restriction</li>
|
||||
<li><strong>R</strong>obust: have error-handling and well-tested utilities out-of-the-box </li>
|
||||
|
|
@ -1035,8 +1035,8 @@
|
|||
<li><strong>T</strong>rustworthy: deploy models that you and society can confidently trust</li>
|
||||
</ul>
|
||||
<h2 id="why-greatai">Why GreatAI?<a class="headerlink" href="#why-greatai" title="Permanent link">#</a></h2>
|
||||
<p>There are other, existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity to use them, do that because they're great.</p>
|
||||
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. <mark>GreatAI is designed to be as simple to use as possible.</mark> Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
|
||||
<p>There are other existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity to use them, do that because they're great.</p>
|
||||
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operational complexity. <mark>GreatAI is designed to be as simple to use as possible.</mark> Its straightforward, high-level API and sensible default configuration make it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively great effort.</p>
|
||||
<div style="display: flex; justify-content: space-evenly; flex-wrap: wrap;">
|
||||
<p><a class="md-button md-button--primary" href="https://pypi.org/project/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg></span> Find it on PyPI</a></p>
|
||||
<p><a class="md-button md-button--primary" href="https://hub.docker.com/repository/docker/schmelczera/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M349.9 236.3h-66.1v-59.4h66.1v59.4zm0-204.3h-66.1v60.7h66.1V32zm78.2 144.8H362v59.4h66.1v-59.4zm-156.3-72.1h-66.1v60.1h66.1v-60.1zm78.1 0h-66.1v60.1h66.1v-60.1zm276.8 100c-14.4-9.7-47.6-13.2-73.1-8.4-3.3-24-16.7-44.9-41.1-63.7l-14-9.3-9.3 14c-18.4 27.8-23.4 73.6-3.7 103.8-8.7 4.7-25.8 11.1-48.4 10.7H2.4c-8.7 50.8 5.8 116.8 44 162.1 37.1 43.9 92.7 66.2 165.4 66.2 157.4 0 273.9-72.5 328.4-204.2 21.4.4 67.6.1 91.3-45.2 1.5-2.5 6.6-13.2 8.5-17.1l-13.3-8.9zm-511.1-27.9h-66v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm-78.1-72.1h-66.1v60.1h66.1v-60.1z"/></svg></span> Find it on DockerHub</a></p>
|
||||
|
|
@ -1052,7 +1052,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 29, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
14
index.md
14
index.md
|
|
@ -10,7 +10,7 @@
|
|||
[](https://sonar.scoutinscience.com/dashboard?id=great-ai)
|
||||
[](https://sonar.scoutinscience.com/dashboard?id=great-ai)
|
||||
|
||||
Applying AI is becoming increasingly easier but many case studies have shown that these applications are often deployed poorly. This may lead to suboptimal performance and to introducing [unintended biases](https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction){ target=_blank }. GreatAI helps fixing this by allowing you to ==easily transform your prototype AI code into production-ready software==.
|
||||
Applying AI is becoming increasingly easier, but many case studies have shown that these applications are often deployed poorly. This may lead to suboptimal performance and to introducing [unintended biases](https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction){ target=_blank }. GreatAI helps fix this by allowing you to ==easily transform your prototype AI code into production-ready software==.
|
||||
|
||||
??? quote "Case studies"
|
||||
"There is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature."
|
||||
|
|
@ -20,11 +20,11 @@ Applying AI is becoming increasingly easier but many case studies have shown tha
|
|||
|
||||
"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." — [Sculley et al.](https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf){ target=_blank }
|
||||
|
||||
"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. — [Serban et al.](https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw){ target=_blank }
|
||||
"For example, practice 25 is very important for "Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. — [Serban et al.](https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw){ target=_blank }
|
||||
|
||||
## Features
|
||||
|
||||
- [x] Save prediction traces of each prediction including arguments and model versions
|
||||
- [x] Save prediction traces of each prediction, including arguments and model versions
|
||||
- [x] Save feedback and merge it into a ground-truth database
|
||||
- [x] Version and store models and data on shared infrastructure *(MongoDB GridFS, S3-compatible storage, shared volume)*
|
||||
- [x] Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration
|
||||
|
|
@ -72,7 +72,7 @@ def greeter(name: str) -> str: #(2)
|
|||
2. [Typing functions](https://docs.python.org/3/library/typing.html){ target=_blank } is recommended in general, however, not required for GreatAI to work.
|
||||
|
||||
??? note
|
||||
In practice, `greeter` could be an inference function of some AI/ML application. But it could also just wrap a black-box solution of some SaaS. Either way, it is [imperative to have continuous oversight](https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai){ target=_blank } of the services you provide and data you process especially in the context of AI/ML applications.
|
||||
In practice, `greeter` could be an inference function of some AI/ML application. But it could also just wrap a black-box solution of some SaaS. Either way, it is [imperative to have continuous oversight](https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai){ target=_blank } of the services you provide and the data you process, especially in the context of AI/ML applications.
|
||||
|
||||
```sh title="terminal"
|
||||
great-ai demo.py
|
||||
|
|
@ -88,7 +88,7 @@ great-ai demo.py
|
|||
|
||||
|
||||
|
||||
GreatAI fits between the prototype and deployment phases of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, several best practices can be automatically implemented aiming to achieve the following attributes:
|
||||
GreatAI fits between the prototype and deployment phases of your (or your organisation's) AI development lifecycle. This is highlighted in blue in the diagram. Here, several best practices can be automatically implemented, aiming to achieve the following attributes:
|
||||
|
||||
- **G**eneral: use any Python library without restriction
|
||||
- **R**obust: have error-handling and well-tested utilities out-of-the-box
|
||||
|
|
@ -98,9 +98,9 @@ GreatAI fits between the prototype and deployment phases of your (or your organi
|
|||
|
||||
## Why GreatAI?
|
||||
|
||||
There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity to use them, do that because they're great.
|
||||
There are other existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity to use them, do that because they're great.
|
||||
|
||||
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. ==GreatAI is designed to be as simple to use as possible.== Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
|
||||
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operational complexity. ==GreatAI is designed to be as simple to use as possible.== Its straightforward, high-level API and sensible default configuration make it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively great effort.
|
||||
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly; flex-wrap: wrap;" markdown>
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
54
sitemap.xml
54
sitemap.xml
|
|
@ -2,137 +2,137 @@
|
|||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-08-19</lastmod>
|
||||
<lastmod>2022-08-20</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
</urlset>
|
||||
BIN
sitemap.xml.gz
BIN
sitemap.xml.gz
Binary file not shown.
|
|
@ -947,7 +947,7 @@
|
|||
|
||||
|
||||
<h1 id="train-and-deploy-a-sota-model">Train and deploy a SOTA model<a class="headerlink" href="#train-and-deploy-a-sota-model" title="Permanent link">#</a></h1>
|
||||
<p>Let's see <code>great-ai</code> in action by going over the life-cycle of a simple service.</p>
|
||||
<p>Let's see <code>great-ai</code> in action by going over the lifecycle of a simple service.</p>
|
||||
<h2 id="objectives">Objectives<a class="headerlink" href="#objectives" title="Permanent link">#</a></h2>
|
||||
<ol>
|
||||
<li>You will see how the <a href="/reference/utilities">great_ai.utilities</a> can integrate into your Data Science workflow.</li>
|
||||
|
|
@ -955,11 +955,11 @@
|
|||
<li>You will use <a class="autorefs autorefs-internal" href="../reference/#great_ai.GreatAI">GreatAI</a> to prepare your model for a robust and responsible deployment.</li>
|
||||
</ol>
|
||||
<h2 id="overview">Overview<a class="headerlink" href="#overview" title="Permanent link">#</a></h2>
|
||||
<p>You are going to train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the <a href="https://arxiv.org/abs/1903.10676" target="_blank">SciBERT paper</a> in which SciBERT <a href="https://paperswithcode.com/sota/sentence-classification-on-paper-field" target="_blank">achieved an F1-score of 0.6571</a>. We are going to outperform it using a trivial text classification model: a <a href="https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html" target="_blank">Linear SVM</a>.</p>
|
||||
<p>You will train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the <a href="https://arxiv.org/abs/1903.10676" target="_blank">SciBERT paper</a> in which SciBERT <a href="https://paperswithcode.com/sota/sentence-classification-on-paper-field" target="_blank">achieved an F1-score of 0.6571</a>. We are going to outperform it using a trivial text classification model: a <a href="https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html" target="_blank">Linear SVM</a>.</p>
|
||||
<p>We use the same synthetic dataset derived from the <a href="https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/" target="_blank">Microsoft Academic Graph</a>. The dataset is <a href="https://github.com/allenai/scibert/tree/master/data/text_classification/mag" target="_blank">available here</a>.</p>
|
||||
<div class="admonition success">
|
||||
<p class="admonition-title">Success</p>
|
||||
<p>You are ready to start the tutorial. Feel free to come back to the <a href="#summary">summary</a> section once you're finished.</p>
|
||||
<p>You are ready to start the tutorial. Feel free to return to the <a href="#summary">summary</a> section once you're finished.</p>
|
||||
</div>
|
||||
<div style="display: flex; justify-content: space-evenly;">
|
||||
<p><a class="md-button md-button--primary" href="train/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M160 80c0-26.51 21.5-48 48-48h32c26.5 0 48 21.49 48 48v352c0 26.5-21.5 48-48 48h-32c-26.5 0-48-21.5-48-48V80zM0 272c0-26.5 21.49-48 48-48h32c26.5 0 48 21.5 48 48v160c0 26.5-21.5 48-48 48H48c-26.51 0-48-21.5-48-48V272zM400 96c26.5 0 48 21.5 48 48v288c0 26.5-21.5 48-48 48h-32c-26.5 0-48-21.5-48-48V144c0-26.5 21.5-48 48-48h32z"/></svg></span> Train it</a></p>
|
||||
|
|
@ -971,7 +971,7 @@
|
|||
<p>After training and evaluating a model, it is exported using <a class="autorefs autorefs-internal" href="../reference/#great_ai.save_model">great_ai.save_model</a>.</p>
|
||||
<details class="tip">
|
||||
<summary>Remote storage</summary>
|
||||
<p>To store your model remotely, you need to set your credentials before calling <code>save_model</code>.</p>
|
||||
<p>To store your model remotely, you must set your credentials before calling <code>save_model</code>.</p>
|
||||
<p>For example, to use <a href="https://aws.amazon.com/s3" target="_blank">AWS S3</a>:
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="kn">from</span> <span class="nn">great_ai.large_file</span> <span class="kn">import</span> <span class="n">LargeFileS3</span>
|
||||
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a>
|
||||
|
|
@ -1001,7 +1001,7 @@
|
|||
</code></pre></div>
|
||||
<ol>
|
||||
<li><a class="autorefs autorefs-internal" href="../reference/#great_ai.use_model">@use_model</a> loads and injects your model into the <code>predict_domain</code> function's <code>model</code> argument.
|
||||
You can freely reference it knowing that the function is always provided with it.</li>
|
||||
You can freely reference it, knowing that the function is always provided with it.</li>
|
||||
</ol>
|
||||
<p>Finally, we test the model's inference function through the GreatAI dashboard. <a href="/how-to-guides/use-service">The only thing left is to deploy the hardened service properly.</a></p>
|
||||
<div style="display: flex; justify-content: space-evenly;">
|
||||
|
|
@ -1014,7 +1014,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">August 20, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Train and deploy a SOTA model
|
||||
|
||||
Let's see `great-ai` in action by going over the life-cycle of a simple service.
|
||||
Let's see `great-ai` in action by going over the lifecycle of a simple service.
|
||||
|
||||
## Objectives
|
||||
|
||||
|
|
@ -10,12 +10,12 @@ Let's see `great-ai` in action by going over the life-cycle of a simple service.
|
|||
|
||||
## Overview
|
||||
|
||||
You are going to train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the [SciBERT paper](https://arxiv.org/abs/1903.10676){ target=_blank } in which SciBERT [achieved an F1-score of 0.6571](https://paperswithcode.com/sota/sentence-classification-on-paper-field){ target=_blank }. We are going to outperform it using a trivial text classification model: a [Linear SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html){ target=_blank }.
|
||||
You will train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the [SciBERT paper](https://arxiv.org/abs/1903.10676){ target=_blank } in which SciBERT [achieved an F1-score of 0.6571](https://paperswithcode.com/sota/sentence-classification-on-paper-field){ target=_blank }. We are going to outperform it using a trivial text classification model: a [Linear SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html){ target=_blank }.
|
||||
|
||||
We use the same synthetic dataset derived from the [Microsoft Academic Graph](https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/){ target=_blank }. The dataset is [available here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag){ target=_blank }.
|
||||
|
||||
!!! success
|
||||
You are ready to start the tutorial. Feel free to come back to the [summary](#summary) section once you're finished.
|
||||
You are ready to start the tutorial. Feel free to return to the [summary](#summary) section once you're finished.
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||
[:fontawesome-solid-chart-simple: Train it](train.ipynb){ .md-button .md-button--primary }
|
||||
|
|
@ -32,7 +32,7 @@ We load and preprocess the dataset while relying on [great_ai.utilities.clean][g
|
|||
After training and evaluating a model, it is exported using [great_ai.save_model][].
|
||||
|
||||
??? tip "Remote storage"
|
||||
To store your model remotely, you need to set your credentials before calling `save_model`.
|
||||
To store your model remotely, you must set your credentials before calling `save_model`.
|
||||
|
||||
For example, to use [AWS S3](https://aws.amazon.com/s3){ target=_blank }:
|
||||
```python
|
||||
|
|
@ -68,7 +68,7 @@ def predict_domain(sentence, model):
|
|||
```
|
||||
|
||||
1. [@use_model][great_ai.use_model] loads and injects your model into the `predict_domain` function's `model` argument.
|
||||
You can freely reference it knowing that the function is always provided with it.
|
||||
You can freely reference it, knowing that the function is always provided with it.
|
||||
|
||||
Finally, we test the model's inference function through the GreatAI dashboard. [The only thing left is to deploy the hardened service properly.](/how-to-guides/use-service)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue