andras/great-ai
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 4 Packages Wiki Activity Actions

Deployed 9999b7e with MkDocs version: 1.3.0

Browse source
This commit is contained in:
2022-07-15 18:04:53 +00:00
parent 087e049514
commit 7d09422380
37 changed files with 116 additions and 108 deletions
Download patch file Download diff file Expand all files Collapse all files

16
how-to-guides/use-service/index.html
View file

@ -540,7 +540,7 @@
<li class="md-nav__item">
<a href="../large_file/" class="md-nav__link">
How to use LargeFile-s
How to use LargeFiles
</a>
</li>
@ -918,7 +918,7 @@
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a>
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a><span class="nd">@GreatAI</span><span class="o">.</span><span class="n">create</span>
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a><span class="k">def</span> <span class="nf">greeter</span><span class="p">(</span><span class="n">your_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a> <span class="k">return</span> <span class="sa">f</span><span class="s1">&#39;Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s1">&#39;</span>
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a> <span class="k">return</span> <span class="sa">f</span><span class="s1">&#39;Hi </span><span class="si">{</span><span class="n">your_name</span><span class="si">}</span><span class="s1">!&#39;</span>
</code></pre></div>
<h2 id="one-off-prediction">One-off prediction<a class="headerlink" href="#one-off-prediction" title="Permanent link">#</a></h2>
<p>Even though <code>greeter</code> is now an instance of <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI</a>, you can continue using it as a regular function.</p>
@ -929,14 +929,14 @@
<a id="__codelineno-1-5" name="__codelineno-1-5" href="#__codelineno-1-5"></a> <span class="s1">&#39;logged_values&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;arg:your_name:length&#39;</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span> <span class="s1">&#39;arg:your_name:value&#39;</span><span class="p">:</span> <span class="s1">&#39;Bob&#39;</span><span class="p">},</span>
<a id="__codelineno-1-6" name="__codelineno-1-6" href="#__codelineno-1-6"></a> <span class="s1">&#39;models&#39;</span><span class="p">:</span> <span class="p">[],</span>
<a id="__codelineno-1-7" name="__codelineno-1-7" href="#__codelineno-1-7"></a> <span class="s1">&#39;original_execution_time_ms&#39;</span><span class="p">:</span> <span class="mf">0.0381</span><span class="p">,</span>
<a id="__codelineno-1-8" name="__codelineno-1-8" href="#__codelineno-1-8"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Bob&#39;</span><span class="p">,</span>
<a id="__codelineno-1-8" name="__codelineno-1-8" href="#__codelineno-1-8"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Bob!&#39;</span><span class="p">,</span>
<a id="__codelineno-1-9" name="__codelineno-1-9" href="#__codelineno-1-9"></a> <span class="s1">&#39;tags&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;greeter&#39;</span><span class="p">,</span> <span class="s1">&#39;online&#39;</span><span class="p">,</span> <span class="s1">&#39;development&#39;</span><span class="p">],</span>
<a id="__codelineno-1-10" name="__codelineno-1-10" href="#__codelineno-1-10"></a> <span class="s1">&#39;trace_id&#39;</span><span class="p">:</span> <span class="s1">&#39;7c284fd7-7f0d-4464-b5f8-3ef126df34af&#39;</span><span class="p">})</span>
</code></pre></div>
<p>As you can see, the original return value is wrapped in a <a class="autorefs autorefs-internal" href="../../reference/views/#great_ai.Trace">Trace</a> object (which is also persisted in your database of choice). You can access the original value under the <code>output</code> property.</p>
<h2 id="online-prediction">Online prediction<a class="headerlink" href="#online-prediction" title="Permanent link">#</a></h2>
<p>Likely, the main way you would like to expose your model is through an HTTP API. <a class="autorefs autorefs-internal" href="../../reference/#great_ai.deploy.great_ai.GreatAI.create">@GreatAI.create</a> scaffolds many REST API endpoints for your model and creates a <a href="https://fastapi.tiangolo.com/" target="_blank">FastAPI</a> app available under <a class="autorefs autorefs-internal" href="../../reference/#great_ai.GreatAI">GreatAI.app</a>. This can be served using <a href="https://www.uvicorn.org/" target="_blank">uvicorn</a> or any other <a href="https://asgi.readthedocs.io/en/latest/" target="_blank">ASGI server</a>.</p>
<p>Since most ML code lives in <a href="https://jupyter.org/" target="_blank">Jupyter</a> notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, <code>uvicorn</code> is wrapped by the <code>great-ai</code> command-line utility which, among others, takes care of feeding a notebook into <code>uvicorn</code>. It also supports auto-reloading.</p>
<p>Since most ML code lives in <a href="https://jupyter.org/" target="_blank">Jupyter</a> notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, <code>uvicorn</code> is wrapped by the <code>great-ai</code> command-line utility which &mdash; among others &mdash; takes care of feeding a notebook into <code>uvicorn</code>. It also supports auto-reloading.</p>
<h3 id="in-development">In development<a class="headerlink" href="#in-development" title="Permanent link">#</a></h3>
<div class="highlight"><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a>great-ai greeter.py
</code></pre></div>
@ -959,7 +959,7 @@
</code></pre></div>
<p>Simply run <code>ENVIRONMENT=production great-ai deploy.ipynb</code> in the command-line of a production machine.</p>
<blockquote>
<p>This is the crudest approach, however, it might be fitting for some contexts.</p>
<p>This is the crudest approach; however, it might be fitting for some contexts.</p>
</blockquote>
<h4 id="containerised-deployment">Containerised deployment<a class="headerlink" href="#containerised-deployment" title="Permanent link">#</a></h4>
<p>Run the notebook directly in a container or create a service for it using your favourite container orchestrator.</p>
@ -970,7 +970,7 @@
<p>You can replace <code>pwd</code> with the path to your code's folder.</p>
</blockquote>
<h4 id="use-a-platform-as-a-service">Use a Platform-as-a-Service<a class="headerlink" href="#use-a-platform-as-a-service" title="Permanent link">#</a></h4>
<p>Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as <a href="https://aws.amazon.com/ecs/" target="_blank">AWS ECS</a>, <a href="https://www.digitalocean.com/products/app-platform" target="_blank">DO App platform</a>, <a href="https://mlem.ai/" target="_blank">MLEM</a>, <a href="https://streamlit.io/" target="_blank">Streamlit</a>) that take a Docker image as a source and handle the rest of the deployment.</p>
<p>Similar to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as <a href="https://aws.amazon.com/ecs/" target="_blank">AWS ECS</a>, <a href="https://www.digitalocean.com/products/app-platform" target="_blank">DO App platform</a>, <a href="https://mlem.ai/" target="_blank">MLEM</a>, <a href="https://streamlit.io/" target="_blank">Streamlit</a>) that take a Docker image as a source and handle the rest of the deployment.</p>
<p>To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as <a href="https://pytorch.org/" target="_blank">PyTorch</a> or <a href="https://www.tensorflow.org/" target="_blank">TensorFlow</a>.</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a><span class="k">FROM</span><span class="w"> </span><span class="s">schmelczera/great-ai:latest</span>
<a id="__codelineno-6-2" name="__codelineno-6-2" href="#__codelineno-6-2"></a>
@ -998,7 +998,7 @@
<a id="__codelineno-7-5" name="__codelineno-7-5" href="#__codelineno-7-5"></a> <span class="s1">&#39;logged_values&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;arg:your_name:length&#39;</span><span class="p">:</span> <span class="mi">5</span><span class="p">,</span> <span class="s1">&#39;arg:your_name:value&#39;</span><span class="p">:</span> <span class="s1">&#39;Alice&#39;</span><span class="p">},</span>
<a id="__codelineno-7-6" name="__codelineno-7-6" href="#__codelineno-7-6"></a> <span class="s1">&#39;models&#39;</span><span class="p">:</span> <span class="p">[],</span>
<a id="__codelineno-7-7" name="__codelineno-7-7" href="#__codelineno-7-7"></a> <span class="s1">&#39;original_execution_time_ms&#39;</span><span class="p">:</span> <span class="mf">0.1251</span><span class="p">,</span>
<a id="__codelineno-7-8" name="__codelineno-7-8" href="#__codelineno-7-8"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Alice&#39;</span><span class="p">,</span>
<a id="__codelineno-7-8" name="__codelineno-7-8" href="#__codelineno-7-8"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Alice!&#39;</span><span class="p">,</span>
<a id="__codelineno-7-9" name="__codelineno-7-9" href="#__codelineno-7-9"></a> <span class="s1">&#39;tags&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;greeter&#39;</span><span class="p">,</span> <span class="s1">&#39;online&#39;</span><span class="p">,</span> <span class="s1">&#39;development&#39;</span><span class="p">],</span>
<a id="__codelineno-7-10" name="__codelineno-7-10" href="#__codelineno-7-10"></a> <span class="s1">&#39;trace_id&#39;</span><span class="p">:</span> <span class="s1">&#39;90ffa15f-e839-41c4-8e7a-3211168bc138&#39;</span><span class="p">}),</span>
<a id="__codelineno-7-11" name="__codelineno-7-11" href="#__codelineno-7-11"></a> <span class="n">Trace</span><span class="p">[</span><span class="nb">str</span><span class="p">]({</span><span class="s1">&#39;created&#39;</span><span class="p">:</span> <span class="s1">&#39;2022-07-11T14:36:37.166659&#39;</span><span class="p">,</span>
@ -1007,7 +1007,7 @@
<a id="__codelineno-7-14" name="__codelineno-7-14" href="#__codelineno-7-14"></a> <span class="s1">&#39;logged_values&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;arg:your_name:length&#39;</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span> <span class="s1">&#39;arg:your_name:value&#39;</span><span class="p">:</span> <span class="s1">&#39;Bob&#39;</span><span class="p">},</span>
<a id="__codelineno-7-15" name="__codelineno-7-15" href="#__codelineno-7-15"></a> <span class="s1">&#39;models&#39;</span><span class="p">:</span> <span class="p">[],</span>
<a id="__codelineno-7-16" name="__codelineno-7-16" href="#__codelineno-7-16"></a> <span class="s1">&#39;original_execution_time_ms&#39;</span><span class="p">:</span> <span class="mf">0.0571</span><span class="p">,</span>
<a id="__codelineno-7-17" name="__codelineno-7-17" href="#__codelineno-7-17"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Bob&#39;</span><span class="p">,</span>
<a id="__codelineno-7-17" name="__codelineno-7-17" href="#__codelineno-7-17"></a> <span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="s1">&#39;Hi Bob!&#39;</span><span class="p">,</span>
<a id="__codelineno-7-18" name="__codelineno-7-18" href="#__codelineno-7-18"></a> <span class="s1">&#39;tags&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;greeter&#39;</span><span class="p">,</span> <span class="s1">&#39;online&#39;</span><span class="p">,</span> <span class="s1">&#39;development&#39;</span><span class="p">],</span>
<a id="__codelineno-7-19" name="__codelineno-7-19" href="#__codelineno-7-19"></a> <span class="s1">&#39;trace_id&#39;</span><span class="p">:</span> <span class="s1">&#39;f48e94c7-0815-48b3-a864-41349d3dae84&#39;</span><span class="p">})]</span>
</code></pre></div>

14
how-to-guides/use-service/use-service.md
View file

@ -9,7 +9,7 @@ from great_ai import GreatAI
@GreatAI.create
def greeter(your_name: str) -> str:
return f'Hi {your_name}'
return f'Hi {your_name}!'
```
## One-off prediction
@ -24,7 +24,7 @@ Trace[str]({'created': '2022-07-11T14:31:46.183764',
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
'models': [],
'original_execution_time_ms': 0.0381,
'output': 'Hi Bob',
'output': 'Hi Bob!',
'tags': ['greeter', 'online', 'development'],
'trace_id': '7c284fd7-7f0d-4464-b5f8-3ef126df34af'})
```
@ -35,7 +35,7 @@ As you can see, the original return value is wrapped in a [Trace][great_ai.Trace
Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, `uvicorn` is wrapped by the `great-ai` command-line utility which, among others, takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To achieve this, `uvicorn` is wrapped by the `great-ai` command-line utility which &mdash; among others &mdash; takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
### In development
@ -67,7 +67,7 @@ ENVIRONMENT=production great-ai greeter.py
```
Simply run `ENVIRONMENT=production great-ai deploy.ipynb` in the command-line of a production machine.
> This is the crudest approach, however, it might be fitting for some contexts.
> This is the crudest approach; however, it might be fitting for some contexts.
#### Containerised deployment
@ -81,7 +81,7 @@ docker run -p 6060:6060 --volume `pwd`:/app --rm \
#### Use a Platform-as-a-Service
Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [AWS ECS](https://aws.amazon.com/ecs/){ target=_blank }, [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank }, [MLEM](https://mlem.ai/){ target=_blank }, [Streamlit](https://streamlit.io/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
Similar to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [AWS ECS](https://aws.amazon.com/ecs/){ target=_blank }, [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank }, [MLEM](https://mlem.ai/){ target=_blank }, [Streamlit](https://streamlit.io/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.
@ -117,7 +117,7 @@ Processing larger amounts of data on a single machine is made easy by the [Great
'logged_values': {'arg:your_name:length': 5, 'arg:your_name:value': 'Alice'},
'models': [],
'original_execution_time_ms': 0.1251,
'output': 'Hi Alice',
'output': 'Hi Alice!',
'tags': ['greeter', 'online', 'development'],
'trace_id': '90ffa15f-e839-41c4-8e7a-3211168bc138'}),
Trace[str]({'created': '2022-07-11T14:36:37.166659',
@ -126,7 +126,7 @@ Processing larger amounts of data on a single machine is made easy by the [Great
'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
'models': [],
'original_execution_time_ms': 0.0571,
'output': 'Hi Bob',
'output': 'Hi Bob!',
'tags': ['greeter', 'online', 'development'],
'trace_id': 'f48e94c7-0815-48b3-a864-41349d3dae84'})]
```