andras/great-ai
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 4 Packages Wiki Activity Actions

Deployed b1a66cb with MkDocs version: 1.3.0

Browse source
This commit is contained in:
2022-07-12 17:19:00 +00:00
parent 0683061983
commit 7305d2ead3
38 changed files with 3566 additions and 2820 deletions
Download patch file Download diff file Expand all files Collapse all files

34
how-to-guides/call-remote/index.html
View file

@ -48,6 +48,19 @@
<meta property="og:title" content="">
<meta property="og:site_name" content="">
<meta property="og:url" content="">
<meta property="og:description" content="Transform your prototype AI code into production-ready software.">
<meta property="og:type" content="">
<meta property="og:image" content=https://great-ai.scoutinscience.com/media/og-image.png>
<style>
.jupyter-wrapper a {
color: var(--md-typeset-a-color) !important;
}
</style>
</head>
@ -351,6 +364,20 @@
<li class="md-nav__item">
<a href="../install/" class="md-nav__link">
Installation guide
</a>
</li>
<li class="md-nav__item">
<a href="../create-service/" class="md-nav__link">
How to create a GreatAI service
@ -601,7 +628,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Simple example: data engineering
</a>
</li>
@ -615,7 +642,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Optimise and train a model
</a>
</li>
@ -629,7 +656,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Hardening and deployment
</a>
</li>
@ -696,6 +723,7 @@
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/call-remote.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>

7
how-to-guides/configure-service/configure-service.md
View file

@ -35,9 +35,9 @@ configure(
The only aspect that cannot be automated is choosing the backing storage for the database and file storage.
Right now, you have 3 options for storing the models and large datasets: [great_ai.large_file.LargeFileLocal][], [great_ai.large_file.LargeFileMongo][], and [great_ai.large_file.LargeFileS3][].
Right now, you have 3 options for storing the models and large datasets: [LargeFileLocal][great_ai.large_file.LargeFileLocal], [LargeFileMongo][great_ai.large_file.LargeFileMongo], and [LargeFileS3][great_ai.large_file.LargeFileS3].
Without explicit configuration, [great_ai.large_file.LargeFileLocal][] is selected by default. This one still version-controls your files but it only stores them in a local path.
Without explicit configuration, [LargeFileLocal][great_ai.large_file.LargeFileLocal] is selected by default. This one still version-controls your files but it only stores them in a local path.
!!! important
If your working directory contains a `mongo.ini` or `s3.ini` file, an attempt is made to auto-configure [LargeFileMongo][great_ai.large_file.LargeFileMongo] or [LargeFileS3][great_ai.large_file.LargeFileS3] respectively.
@ -76,6 +76,7 @@ save_model(model, 'my-model')
MONGO_CONNECTION_STRING=mongodb://localhost:27017 # this is the default value
# if `MONGO_CONNECTION_STRING` is specified, this default is overridden
MONGO_CONNECTION_STRING=ENV:MONGO_CONNECTION_STRING
MONGO_DATABASE=my-database # it is automatically created if doesn't exist
```
@ -98,4 +99,4 @@ By default, a thread-safe version of [TinyDB](https://tinydb.readthedocs.io/en/l
### MongoDB
At the moment, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory, or explicitly call [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver.configure_credentials_from_file] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].
At the moment, only MongoDB is supported as a production-ready `TracingDatabase`. In order to use it, you have to either place a file named `mongo.ini` in your working directory, or explicitly call [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver] or [MongoDbDriver.configure_credentials][great_ai.MongoDbDriver.configure_credentials].

45
how-to-guides/configure-service/index.html
View file

@ -48,6 +48,19 @@
<meta property="og:title" content="">
<meta property="og:site_name" content="">
<meta property="og:url" content="">
<meta property="og:description" content="Transform your prototype AI code into production-ready software.">
<meta property="og:type" content="">
<meta property="og:image" content=https://great-ai.scoutinscience.com/media/og-image.png>
<style>
.jupyter-wrapper a {
color: var(--md-typeset-a-color) !important;
}
</style>
</head>
@ -353,6 +366,20 @@
<li class="md-nav__item">
<a href="../install/" class="md-nav__link">
Installation guide
</a>
</li>
<li class="md-nav__item">
<a href="../create-service/" class="md-nav__link">
How to create a GreatAI service
@ -691,7 +718,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Simple example: data engineering
</a>
</li>
@ -705,7 +732,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Optimise and train a model
</a>
</li>
@ -719,7 +746,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Hardening and deployment
</a>
</li>
@ -848,6 +875,7 @@
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/configure-service.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
@ -886,8 +914,8 @@
</ol>
<h2 id="using-remote-storage">Using remote storage<a class="headerlink" href="#using-remote-storage" title="Permanent link">#</a></h2>
<p>The only aspect that cannot be automated is choosing the backing storage for the database and file storage.</p>
<p>Right now, you have 3 options for storing the models and large datasets: <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileLocal">great_ai.large_file.LargeFileLocal</a>, <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileMongo">great_ai.large_file.LargeFileMongo</a>, and <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileS3">great_ai.large_file.LargeFileS3</a>.</p>
<p>Without explicit configuration, <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileLocal">great_ai.large_file.LargeFileLocal</a> is selected by default. This one still version-controls your files but it only stores them in a local path.</p>
<p>Right now, you have 3 options for storing the models and large datasets: <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileLocal">LargeFileLocal</a>, <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileMongo">LargeFileMongo</a>, and <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileS3">LargeFileS3</a>.</p>
<p>Without explicit configuration, <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileLocal">LargeFileLocal</a> is selected by default. This one still version-controls your files but it only stores them in a local path.</p>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>If your working directory contains a <code>mongo.ini</code> or <code>s3.ini</code> file, an attempt is made to auto-configure <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileMongo">LargeFileMongo</a> or <a class="autorefs autorefs-internal" href="../../reference/large-file/#great_ai.large_file.LargeFileS3">LargeFileS3</a> respectively.</p>
@ -919,7 +947,8 @@
<div class="highlight"><span class="filename">mongo.ini</span><pre><span></span><code><a id="__codelineno-3-1" name="__codelineno-3-1" href="#__codelineno-3-1"></a><span class="n">MONGO_CONNECTION_STRING</span><span class="o">=</span><span class="n">mongodb</span><span class="p">:</span><span class="err">//</span><span class="n">localhost</span><span class="p">:</span><span class="mi">27017</span><span class="w"> </span><span class="c1"># this is the default value</span><span class="w"></span>
<a id="__codelineno-3-2" name="__codelineno-3-2" href="#__codelineno-3-2"></a><span class="c1"># if `MONGO_CONNECTION_STRING` is specified, this default is overridden</span><span class="w"></span>
<a id="__codelineno-3-3" name="__codelineno-3-3" href="#__codelineno-3-3"></a><span class="n">MONGO_CONNECTION_STRING</span><span class="o">=</span><span class="n">ENV</span><span class="p">:</span><span class="n">MONGO_CONNECTION_STRING</span><span class="w"></span>
<a id="__codelineno-3-4" name="__codelineno-3-4" href="#__codelineno-3-4"></a><span class="n">MONGO_DATABASE</span><span class="o">=</span><span class="n">my-database</span><span class="w"> </span><span class="c1"># it is automatically created if doesn&#39;t exist</span><span class="w"></span>
<a id="__codelineno-3-4" name="__codelineno-3-4" href="#__codelineno-3-4"></a>
<a id="__codelineno-3-5" name="__codelineno-3-5" href="#__codelineno-3-5"></a><span class="n">MONGO_DATABASE</span><span class="o">=</span><span class="n">my-database</span><span class="w"> </span><span class="c1"># it is automatically created if doesn&#39;t exist</span><span class="w"></span>
</code></pre></div>
<div class="highlight"><span class="filename">use-mongo.py</span><pre><span></span><code><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="kn">from</span> <span class="nn">great_ai.large_file</span> <span class="kn">import</span> <span class="n">LargeFileMongo</span>
<a id="__codelineno-4-2" name="__codelineno-4-2" href="#__codelineno-4-2"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">save_model</span>
@ -936,14 +965,14 @@
<h2 id="using-a-database">Using a database<a class="headerlink" href="#using-a-database" title="Permanent link">#</a></h2>
<p>By default, a thread-safe version of <a href="https://tinydb.readthedocs.io/en/latest/" target="_blank">TinyDB</a> is utilised for saving the prediction traces into a local file. Unfortunately, for most production needs, this method is not suitable.</p>
<h3 id="mongodb">MongoDB<a class="headerlink" href="#mongodb" title="Permanent link">#</a></h3>
<p>At the moment, only MongoDB is supported as a production-ready <code>TracingDatabase</code>. In order to use it, you have to either place a file named <code>mongo.ini</code> in your working directory, or explicitly call [MongoDbDriver.configure_credentials_from_file][great_ai.MongoDbDriver.configure_credentials_from_file] or <a class="autorefs autorefs-internal" href="../../reference/#great_ai.persistence.mongodb_driver.MongoDbDriver.configure_credentials">MongoDbDriver.configure_credentials</a>.</p>
<p>At the moment, only MongoDB is supported as a production-ready <code>TracingDatabase</code>. In order to use it, you have to either place a file named <code>mongo.ini</code> in your working directory, or explicitly call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.MongoDbDriver">MongoDbDriver.configure_credentials_from_file</a> or <a class="autorefs autorefs-internal" href="../../reference/#great_ai.persistence.mongodb_driver.MongoDbDriver.configure_credentials">MongoDbDriver.configure_credentials</a>.</p>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 11, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 12, 2022</span>
</small>

8
how-to-guides/create-service/create-service.md
View file

@ -24,7 +24,7 @@ def greeter(your_name):
## With types
[Type annotating your codebase](https://realpython.com/python-type-checking/){ target=_blank } can save you from lots of trivial mistakes, that's why it's highly advised. Simply add the expected types to your function's signature.
Even though it's not required by GreatAI, [type annotating your codebase](https://realpython.com/python-type-checking/){ target=_blank } can save you from lots of trivial mistakes, that's why it's highly advised. Simply add the expected types to your function's signature.
```python title="type_safe_greeter.py"
from great_ai import GreatAI
@ -100,7 +100,7 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
Refer to the following example summarising the options you have when instantiating a GreatAI service.
```python title="complex.py"
from great_ai import save_model, GreatAI, parameter, use_model
from great_ai import save_model, GreatAI, parameter, use_model, log_metric
save_model(4, 'secret-number') #(1)
@ -108,6 +108,10 @@ save_model(4, 'secret-number') #(1)
@parameter('positive_number', validator=lambda n: n > 0, disable_logging=True)
@use_model('secret-number', version='latest', model_kwarg_name='secret')
def add_number(positive_number: int, secret: int) -> int:
log_metric(
'log directly into the returned Trace',
positive_number * 2
)
return positive_number + secret
assert add_number(1).output == 5

54
how-to-guides/create-service/index.html
View file

@ -48,6 +48,19 @@
<meta property="og:title" content="">
<meta property="og:site_name" content="">
<meta property="og:url" content="">
<meta property="og:description" content="Transform your prototype AI code into production-ready software.">
<meta property="og:type" content="">
<meta property="og:image" content=https://great-ai.scoutinscience.com/media/og-image.png>
<style>
.jupyter-wrapper a {
color: var(--md-typeset-a-color) !important;
}
</style>
</head>
@ -352,6 +365,20 @@
<li class="md-nav__item">
<a href="../install/" class="md-nav__link">
Installation guide
</a>
</li>
@ -685,7 +712,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Simple example: data engineering
</a>
</li>
@ -699,7 +726,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Optimise and train a model
</a>
</li>
@ -713,7 +740,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Hardening and deployment
</a>
</li>
@ -836,6 +863,7 @@
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/create-service.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
@ -861,7 +889,7 @@
<p>The purpose of the <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for <code>__new__</code> is lacking in MyPy, thus, a static factory method is used instead.</p>
</details>
<h2 id="with-types">With types<a class="headerlink" href="#with-types" title="Permanent link">#</a></h2>
<p><a href="https://realpython.com/python-type-checking/" target="_blank">Type annotating your codebase</a> can save you from lots of trivial mistakes, that's why it's highly advised. Simply add the expected types to your function's signature.</p>
<p>Even though it's not required by GreatAI, <a href="https://realpython.com/python-type-checking/" target="_blank">type annotating your codebase</a> can save you from lots of trivial mistakes, that's why it's highly advised. Simply add the expected types to your function's signature.</p>
<div class="highlight"><span class="filename">type_safe_greeter.py</span><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span>
<a id="__codelineno-2-2" name="__codelineno-2-2" href="#__codelineno-2-2"></a>
<a id="__codelineno-2-3" name="__codelineno-2-3" href="#__codelineno-2-3"></a><span class="nd">@GreatAI</span><span class="o">.</span><span class="n">create</span>
@ -920,7 +948,7 @@
</div>
<h2 id="complex-example">Complex example<a class="headerlink" href="#complex-example" title="Permanent link">#</a></h2>
<p>Refer to the following example summarising the options you have when instantiating a GreatAI service.</p>
<div class="highlight"><span class="filename">complex.py</span><pre><span></span><code><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">save_model</span><span class="p">,</span> <span class="n">GreatAI</span><span class="p">,</span> <span class="n">parameter</span><span class="p">,</span> <span class="n">use_model</span>
<div class="highlight"><span class="filename">complex.py</span><pre><span></span><code><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">save_model</span><span class="p">,</span> <span class="n">GreatAI</span><span class="p">,</span> <span class="n">parameter</span><span class="p">,</span> <span class="n">use_model</span><span class="p">,</span> <span class="n">log_metric</span>
<a id="__codelineno-6-2" name="__codelineno-6-2" href="#__codelineno-6-2"></a>
<a id="__codelineno-6-3" name="__codelineno-6-3" href="#__codelineno-6-3"></a><span class="n">save_model</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="s1">&#39;secret-number&#39;</span><span class="p">)</span> <span class="c1">#(1)</span>
<a id="__codelineno-6-4" name="__codelineno-6-4" href="#__codelineno-6-4"></a>
@ -928,9 +956,13 @@
<a id="__codelineno-6-6" name="__codelineno-6-6" href="#__codelineno-6-6"></a><span class="nd">@parameter</span><span class="p">(</span><span class="s1">&#39;positive_number&#39;</span><span class="p">,</span> <span class="n">validator</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">,</span> <span class="n">disable_logging</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<a id="__codelineno-6-7" name="__codelineno-6-7" href="#__codelineno-6-7"></a><span class="nd">@use_model</span><span class="p">(</span><span class="s1">&#39;secret-number&#39;</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="s1">&#39;latest&#39;</span><span class="p">,</span> <span class="n">model_kwarg_name</span><span class="o">=</span><span class="s1">&#39;secret&#39;</span><span class="p">)</span>
<a id="__codelineno-6-8" name="__codelineno-6-8" href="#__codelineno-6-8"></a><span class="k">def</span> <span class="nf">add_number</span><span class="p">(</span><span class="n">positive_number</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">secret</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<a id="__codelineno-6-9" name="__codelineno-6-9" href="#__codelineno-6-9"></a> <span class="k">return</span> <span class="n">positive_number</span> <span class="o">+</span> <span class="n">secret</span>
<a id="__codelineno-6-10" name="__codelineno-6-10" href="#__codelineno-6-10"></a>
<a id="__codelineno-6-11" name="__codelineno-6-11" href="#__codelineno-6-11"></a><span class="k">assert</span> <span class="n">add_number</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">output</span> <span class="o">==</span> <span class="mi">5</span>
<a id="__codelineno-6-9" name="__codelineno-6-9" href="#__codelineno-6-9"></a> <span class="n">log_metric</span><span class="p">(</span>
<a id="__codelineno-6-10" name="__codelineno-6-10" href="#__codelineno-6-10"></a> <span class="s1">&#39;log directly into the returned Trace&#39;</span><span class="p">,</span>
<a id="__codelineno-6-11" name="__codelineno-6-11" href="#__codelineno-6-11"></a> <span class="n">positive_number</span> <span class="o">*</span> <span class="mi">2</span>
<a id="__codelineno-6-12" name="__codelineno-6-12" href="#__codelineno-6-12"></a> <span class="p">)</span>
<a id="__codelineno-6-13" name="__codelineno-6-13" href="#__codelineno-6-13"></a> <span class="k">return</span> <span class="n">positive_number</span> <span class="o">+</span> <span class="n">secret</span>
<a id="__codelineno-6-14" name="__codelineno-6-14" href="#__codelineno-6-14"></a>
<a id="__codelineno-6-15" name="__codelineno-6-15" href="#__codelineno-6-15"></a><span class="k">assert</span> <span class="n">add_number</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">output</span> <span class="o">==</span> <span class="mi">5</span>
</code></pre></div>
<ol>
<li>Refer to <a href="/how-to-guides/store-models">storing models</a> for specifying where to store your models. </li>
@ -941,7 +973,7 @@
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 11, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 12, 2022</span>
</small>
@ -962,7 +994,7 @@
<nav class="md-footer__inner md-grid" aria-label="Footer" >
<a href="../../tutorial/deploy/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Harden and deploy your app" rel="prev">
<a href="../install/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Installation guide" rel="prev">
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</div>
@ -971,7 +1003,7 @@
<span class="md-footer__direction">
Previous
</span>
Harden and deploy your app
Installation guide
</div>
</div>
</a>

34
how-to-guides/handle-training-data/index.html
View file

@ -48,6 +48,19 @@
<meta property="og:title" content="">
<meta property="og:site_name" content="">
<meta property="og:url" content="">
<meta property="og:description" content="Transform your prototype AI code into production-ready software.">
<meta property="og:type" content="">
<meta property="og:image" content=https://great-ai.scoutinscience.com/media/og-image.png>
<style>
.jupyter-wrapper a {
color: var(--md-typeset-a-color) !important;
}
</style>
</head>
@ -353,6 +366,20 @@
<li class="md-nav__item">
<a href="../install/" class="md-nav__link">
Installation guide
</a>
</li>
<li class="md-nav__item">
<a href="../create-service/" class="md-nav__link">
How to create a GreatAI service
@ -651,7 +678,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Simple example: data engineering
</a>
</li>
@ -665,7 +692,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Optimise and train a model
</a>
</li>
@ -679,7 +706,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Hardening and deployment
</a>
</li>
@ -768,6 +795,7 @@
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/handle-training-data.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>

924
how-to-guides/install/index.html Normal file
View file

@ -0,0 +1,924 @@
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="GreatAI helps you easily transform your prototype AI code into production-ready software.">
<meta name="author" content="András Schmelczer">
<link rel="icon" href="../../media/favicon.ico">
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
<title>Installation guide - GreatAI documentation</title>
<link rel="stylesheet" href="../../assets/stylesheets/main.1d29e8d0.min.css">
<link rel="stylesheet" href="../../assets/stylesheets/palette.cbb835fc.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<link rel="stylesheet" href="../../assets/_mkdocstrings.css">
<script>__md_scope=new URL("../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
<meta property="og:title" content="">
<meta property="og:site_name" content="">
<meta property="og:url" content="">
<meta property="og:description" content="Transform your prototype AI code into production-ready software.">
<meta property="og:type" content="">
<meta property="og:image" content=https://great-ai.scoutinscience.com/media/og-image.png>
<style>
.jupyter-wrapper a {
color: var(--md-typeset-a-color) !important;
}
</style>
</head>
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="light-blue" data-md-color-accent="">
<script>var palette=__md_get("__palette");if(palette&&"object"==typeof palette.color)for(var key of Object.keys(palette.color))document.body.setAttribute("data-md-color-"+key,palette.color[key])</script>
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#installation-guide" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
GreatAI documentation
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
Installation guide
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme="default" data-md-color-primary="light-blue" data-md-color-accent="" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_2" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 2a7 7 0 0 0-7 7c0 2.38 1.19 4.47 3 5.74V17a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1v-2.26c1.81-1.27 3-3.36 3-5.74a7 7 0 0 0-7-7M9 21a1 1 0 0 0 1 1h4a1 1 0 0 0 1-1v-1H9v1Z"/></svg>
</label>
<input class="md-option" data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme="slate" data-md-color-primary="light-blue" data-md-color-accent="" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_2">
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 2a7 7 0 0 1 7 7c0 2.38-1.19 4.47-3 5.74V17a1 1 0 0 1-1 1H9a1 1 0 0 1-1-1v-2.26C6.19 13.47 5 11.38 5 9a7 7 0 0 1 7-7M9 21v-1h6v1a1 1 0 0 1-1 1h-4a1 1 0 0 1-1-1m3-17a5 5 0 0 0-5 5c0 2.05 1.23 3.81 3 4.58V16h4v-2.42c1.77-.77 3-2.53 3-4.58a5 5 0 0 0-5-5Z"/></svg>
</label>
</form>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41Z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://github.com/schmelczer/great-ai" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
</div>
<div class="md-source__repository">
schmelczer/great-ai
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
</a>
GreatAI documentation
</label>
<div class="md-nav__source">
<a href="https://github.com/schmelczer/great-ai" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
</div>
<div class="md-source__repository">
schmelczer/great-ai
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../.." class="md-nav__link">
Overview
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
<label class="md-nav__link" for="__nav_2">
Tutorial
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
<label class="md-nav__title" for="__nav_2">
<span class="md-nav__icon md-icon"></span>
Tutorial
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../tutorial/" class="md-nav__link">
Tutorial overview
</a>
</li>
<li class="md-nav__item">
<a href="../../tutorial/train/" class="md-nav__link">
Train your model
</a>
</li>
<li class="md-nav__item">
<a href="../../tutorial/deploy/" class="md-nav__link">
Harden and deploy your app
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" checked>
<label class="md-nav__link" for="__nav_3">
User Guides
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
User Guides
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--active">
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
<label class="md-nav__link md-nav__link--active" for="__toc">
Installation guide
<span class="md-nav__icon md-icon"></span>
</label>
<a href="./" class="md-nav__link md-nav__link--active">
Installation guide
</a>
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#command-line-tools" class="md-nav__link">
Command-line tools
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../create-service/" class="md-nav__link">
How to create a GreatAI service
</a>
</li>
<li class="md-nav__item">
<a href="../configure-service/" class="md-nav__link">
How to configure GreatAI
</a>
</li>
<li class="md-nav__item">
<a href="../use-service/" class="md-nav__link">
How to use a GreatAI service
</a>
</li>
<li class="md-nav__item">
<a href="../handle-training-data/" class="md-nav__link">
How to handle training data
</a>
</li>
<li class="md-nav__item">
<a href="../large_file/" class="md-nav__link">
How to use LargeFile
</a>
</li>
<li class="md-nav__item">
<a href="../call_remote.md" class="md-nav__link">
None
</a>
</li>
<li class="md-nav__item">
<a href="../scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_4" type="checkbox" id="__nav_4" >
<label class="md-nav__link" for="__nav_4">
Reference
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Reference" data-md-level="1">
<label class="md-nav__title" for="__nav_4">
<span class="md-nav__icon md-icon"></span>
Reference
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../reference/" class="md-nav__link">
GreatAI reference
</a>
</li>
<li class="md-nav__item">
<a href="../../reference/utilities/" class="md-nav__link">
Utilities
</a>
</li>
<li class="md-nav__item">
<a href="../../reference/large-file/" class="md-nav__link">
LargeFile
</a>
</li>
<li class="md-nav__item">
<a href="../../reference/views/" class="md-nav__link">
View models
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5" type="checkbox" id="__nav_5" >
<label class="md-nav__link" for="__nav_5">
Examples
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Examples" data-md-level="1">
<label class="md-nav__title" for="__nav_5">
<span class="md-nav__icon md-icon"></span>
Examples
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_5_1" type="checkbox" id="__nav_5_1" >
<label class="md-nav__link" for="__nav_5_1">
Explainable Naive Bayes
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Explainable Naive Bayes" data-md-level="2">
<label class="md-nav__title" for="__nav_5_1">
<span class="md-nav__icon md-icon"></span>
Explainable Naive Bayes
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../examples/simple/data/" class="md-nav__link">
Simple example: data engineering
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/simple/train/" class="md-nav__link">
Optimise and train a model
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/simple/deploy/" class="md-nav__link">
Hardening and deployment
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../../explanation/" class="md-nav__link">
Explanation
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#command-line-tools" class="md-nav__link">
Command-line tools
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/install.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
</a>
<h1 id="installation-guide">Installation guide<a class="headerlink" href="#installation-guide" title="Permanent link">#</a></h1>
<p>Provided you already have <a href="https://www.python.org/downloads/" target="_blank">Python3</a> (and pip) installed, simply execute:</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a>pip install great-ai
</code></pre></div>
<blockquote>
<p>Python 3.8 or later is required.</p>
</blockquote>
<p>This will work on all major operating systems.</p>
<h2 id="command-line-tools">Command-line tools<a class="headerlink" href="#command-line-tools" title="Permanent link">#</a></h2>
<p>After installation, <code>great-ai</code> and <code>large-file</code> are available as commands. The former is required for deploying your application while the latter lets you manage models and datasets from your terminal.</p>
<div class="admonition warning">
<p class="admonition-title">Windows</p>
<p>On Windows, you might encounter a similar <mark>warning</mark> from <code>pip</code>:</p>
<blockquote>
<p>WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users...\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.</p>
</blockquote>
<p>This means that <code>great-ai.exe</code> and <code>large-file.exe</code> are not in your <code>PATH</code>. Either add their containing directory ('C:\Users...\Scripts' in this case) to your <code>PATH</code> or use <code>python3 -m great_ai</code> and <code>python3 -m great_ai.large_file</code> instead of the exe-s.</p>
</div>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 12, 2022</span>
</small>
</div>
</article>
</div>
</div>
</main>
<footer class="md-footer">
<nav class="md-footer__inner md-grid" aria-label="Footer" >
<a href="../../tutorial/deploy/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Harden and deploy your app" rel="prev">
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</div>
<div class="md-footer__title">
<div class="md-ellipsis">
<span class="md-footer__direction">
Previous
</span>
Harden and deploy your app
</div>
</div>
</a>
<a href="../create-service/" class="md-footer__link md-footer__link--next" aria-label="Next: How to create a GreatAI service" rel="next">
<div class="md-footer__title">
<div class="md-ellipsis">
<span class="md-footer__direction">
Next
</span>
How to create a GreatAI service
</div>
</div>
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
</div>
</a>
</nav>
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
<div class="md-copyright__highlight">
GNU General Public License v3
</div>
</div>
<div class="md-social">
<a href="https://schmelczer.dev" target="_blank" rel="noopener" title="about the author" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="m414.8 40.79-128 448.01c-4.9 17-22.6 26.8-39.6 22-17-4.9-26.8-22.6-22-39.6l128-447.99c4.9-16.994 22.6-26.834 39.6-21.978 17 4.855 26.8 22.568 22 39.558zm103.8 80.61 112 112c12.5 12.5 12.5 32.7 0 45.2l-112 112c-12.5 12.5-32.7 12.5-45.2 0s-12.5-32.7 0-45.2l89.3-89.4-89.3-89.4c-12.5-12.5-12.5-32.7 0-45.2s32.7-12.5 45.2 0zm-352 45.2L77.25 256l89.35 89.4c12.5 12.5 12.5 32.7 0 45.2s-32.7 12.5-45.2 0L9.372 278.6c-12.496-12.5-12.496-32.7 0-45.2l112.028-112c12.5-12.5 32.7-12.5 45.2 0s12.5 32.7 0 45.2z"/></svg>
</a>
<a href="https://github.com/schmelczer/great-ai" target="_blank" rel="noopener" title="great-ai on Github" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 496 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg>
</a>
<a href="https://hub.docker.com/repository/docker/schmelczera/great-ai" target="_blank" rel="noopener" title="great-ai on DockerHub" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M349.9 236.3h-66.1v-59.4h66.1v59.4zm0-204.3h-66.1v60.7h66.1V32zm78.2 144.8H362v59.4h66.1v-59.4zm-156.3-72.1h-66.1v60.1h66.1v-60.1zm78.1 0h-66.1v60.1h66.1v-60.1zm276.8 100c-14.4-9.7-47.6-13.2-73.1-8.4-3.3-24-16.7-44.9-41.1-63.7l-14-9.3-9.3 14c-18.4 27.8-23.4 73.6-3.7 103.8-8.7 4.7-25.8 11.1-48.4 10.7H2.4c-8.7 50.8 5.8 116.8 44 162.1 37.1 43.9 92.7 66.2 165.4 66.2 157.4 0 273.9-72.5 328.4-204.2 21.4.4 67.6.1 91.3-45.2 1.5-2.5 6.6-13.2 8.5-17.1l-13.3-8.9zm-511.1-27.9h-66v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm-78.1-72.1h-66.1v60.1h66.1v-60.1z"/></svg>
</a>
<a href="https://pypi.org/project/great-ai" target="_blank" rel="noopener" title="great-ai on PyPI" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg>
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"base": "../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
<script src="../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
</body>
</html>

21
how-to-guides/install/install.md Normal file
View file

@ -0,0 +1,21 @@
# Installation guide
Provided you already have [Python3](https://www.python.org/downloads/){ target=_blank } (and pip) installed, simply execute:
```sh
pip install great-ai
```
> Python 3.8 or later is required.
This will work on all major operating systems.
## Command-line tools
After installation, `great-ai` and `large-file` are available as commands. The former is required for deploying your application while the latter lets you manage models and datasets from your terminal.
!!! warning "Windows"
On Windows, you might encounter a similar ==warning== from `pip`:
> WARNING: The scripts great-ai.exe, great_ai.exe, large-file.exe and large_file.exe are installed in 'C:\Users\...\Scripts' which is not on PATH.
> Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
This means that `great-ai.exe` and `large-file.exe` are not in your `PATH`. Either add their containing directory ('C:\Users\...\Scripts' in this case) to your `PATH` or use `python3 -m great_ai` and `python3 -m great_ai.large_file` instead of the exe-s.

34
how-to-guides/large_file/index.html
View file

@ -48,6 +48,19 @@
<meta property="og:title" content="">
<meta property="og:site_name" content="">
<meta property="og:url" content="">
<meta property="og:description" content="Transform your prototype AI code into production-ready software.">
<meta property="og:type" content="">
<meta property="og:image" content=https://great-ai.scoutinscience.com/media/og-image.png>
<style>
.jupyter-wrapper a {
color: var(--md-typeset-a-color) !important;
}
</style>
</head>
@ -353,6 +366,20 @@
<li class="md-nav__item">
<a href="../install/" class="md-nav__link">
Installation guide
</a>
</li>
<li class="md-nav__item">
<a href="../create-service/" class="md-nav__link">
How to create a GreatAI service
@ -732,7 +759,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Simple example: data engineering
</a>
</li>
@ -746,7 +773,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Optimise and train a model
</a>
</li>
@ -760,7 +787,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Hardening and deployment
</a>
</li>
@ -930,6 +957,7 @@
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/large_file.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>

34
how-to-guides/scraping/index.html
View file

@ -48,6 +48,19 @@
<meta property="og:title" content="">
<meta property="og:site_name" content="">
<meta property="og:url" content="">
<meta property="og:description" content="Transform your prototype AI code into production-ready software.">
<meta property="og:type" content="">
<meta property="og:image" content=https://great-ai.scoutinscience.com/media/og-image.png>
<style>
.jupyter-wrapper a {
color: var(--md-typeset-a-color) !important;
}
</style>
</head>
@ -348,6 +361,20 @@
<li class="md-nav__item">
<a href="../install/" class="md-nav__link">
Installation guide
</a>
</li>
<li class="md-nav__item">
<a href="../create-service/" class="md-nav__link">
How to create a GreatAI service
@ -606,7 +633,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Simple example: data engineering
</a>
</li>
@ -620,7 +647,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Optimise and train a model
</a>
</li>
@ -634,7 +661,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Hardening and deployment
</a>
</li>
@ -697,6 +724,7 @@
<article class="md-content__inner md-typeset">
<a href="scraping.ipynb" title="Download Notebook" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M5 20h14v-2H5m14-9h-4V3H9v6H5l7 7 7-7Z"/></svg>
</a>

45
how-to-guides/use-service/index.html
View file

@ -48,6 +48,19 @@
<meta property="og:title" content="">
<meta property="og:site_name" content="">
<meta property="og:url" content="">
<meta property="og:description" content="Transform your prototype AI code into production-ready software.">
<meta property="og:type" content="">
<meta property="og:image" content=https://great-ai.scoutinscience.com/media/og-image.png>
<style>
.jupyter-wrapper a {
color: var(--md-typeset-a-color) !important;
}
</style>
</head>
@ -353,6 +366,20 @@
<li class="md-nav__item">
<a href="../install/" class="md-nav__link">
Installation guide
</a>
</li>
<li class="md-nav__item">
<a href="../create-service/" class="md-nav__link">
How to create a GreatAI service
@ -705,7 +732,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Simple example: data engineering
</a>
</li>
@ -719,7 +746,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Optimise and train a model
</a>
</li>
@ -733,7 +760,7 @@
<li class="md-nav__item">
<a href="../../examples/simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
Hardening and deployment
</a>
</li>
@ -876,6 +903,7 @@
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/use-service.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
@ -919,9 +947,10 @@
<p>Some configuration options are also supported.</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-3-1" name="__codelineno-3-1" href="#__codelineno-3-1"></a>great-ai greeter.py --port <span class="m">8000</span> --host <span class="m">127</span>.0.0.1 --timeout_keep_alive <span class="m">10</span>
</code></pre></div>
<blockquote>
<p>For more options (but no Notebook support, use <a href="https://www.uvicorn.org/" target="_blank">uvicorn</a>)</p>
</blockquote>
<details class="note">
<summary>More options</summary>
<p>For more options (but no Notebook support), simply use <a href="https://www.uvicorn.org/" target="_blank">uvicorn</a> for starting your app (available at <code>greeter.app</code>).</p>
</details>
<h3 id="in-production">In production<a class="headerlink" href="#in-production" title="Permanent link">#</a></h3>
<p>There are three main approaches for deploying a GreatAI service.</p>
<h4 id="manual-deployment">Manual deployment<a class="headerlink" href="#manual-deployment" title="Permanent link">#</a></h4>
@ -941,7 +970,7 @@
<p>You can replace <code>pwd</code> with the path to your code's folder.</p>
</blockquote>
<h4 id="use-a-platform-as-a-service">Use a Platform-as-a-Service<a class="headerlink" href="#use-a-platform-as-a-service" title="Permanent link">#</a></h4>
<p>Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as <a href="https://www.digitalocean.com/products/app-platform" target="_blank">DO App platform</a> or <a href="https://mlem.ai/" target="_blank">MLEM</a>) that take a Docker image as a source and handle the rest of the deployment.</p>
<p>Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as <a href="https://aws.amazon.com/ecs/" target="_blank">AWS ECS</a>, <a href="https://www.digitalocean.com/products/app-platform" target="_blank">DO App platform</a>, <a href="https://mlem.ai/" target="_blank">MLEM</a>) that take a Docker image as a source and handle the rest of the deployment.</p>
<p>To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as <a href="https://pytorch.org/" target="_blank">PyTorch</a> or <a href="https://www.tensorflow.org/" target="_blank">TensorFlow</a>.</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a><span class="k">FROM</span><span class="w"> </span><span class="s">schmelczera/great-ai:latest</span>
<a id="__codelineno-6-2" name="__codelineno-6-2" href="#__codelineno-6-2"></a>
@ -988,7 +1017,7 @@
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 11, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 12, 2022</span>
</small>

5
how-to-guides/use-service/use-service.md
View file

@ -51,7 +51,8 @@ Some configuration options are also supported.
```sh
great-ai greeter.py --port 8000 --host 127.0.0.1 --timeout_keep_alive 10
```
> For more options (but no Notebook support, use [uvicorn](https://www.uvicorn.org/){ target=_blank })
??? note "More options"
For more options (but no Notebook support), simply use [uvicorn](https://www.uvicorn.org/){ target=_blank } for starting your app (available at `greeter.app`).
### In production
@ -80,7 +81,7 @@ docker run -p 6060:6060 --volume `pwd`:/app --rm \
#### Use a Platform-as-a-Service
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank } or [MLEM](https://mlem.ai/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [AWS ECS](https://aws.amazon.com/ecs/){ target=_blank }, [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank }, [MLEM](https://mlem.ai/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.