Deployed e8b07b4 with MkDocs version: 1.3.1
This commit is contained in:
parent
f2faa76f96
commit
f254b85515
47 changed files with 146 additions and 128 deletions
2
404.html
2
404.html
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="/media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"source": [
|
||||
"# Simple example: data engineering\n",
|
||||
"\n",
|
||||
"Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best-practices. In short, we train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus) by taking full advantage of `great-ai`. Subsequently, we create a production-ready deployment.\n",
|
||||
"Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best practices. In short, we train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus) by taking full advantage of `great-ai`. Subsequently, we create a production-ready deployment.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"> The blue boxes show the steps of a typical AI-development lifecycle implemented in this notebook.\n",
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
@ -1988,7 +1988,7 @@ Licensed under the Apache License, Version 2.0.
|
|||
</div>
|
||||
<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
|
||||
</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput " data-mime-type="text/markdown">
|
||||
<h1 id="simple-example-data-engineering">Simple example: data engineering<a class="anchor-link" href="#simple-example-data-engineering">¶</a></h1><p>Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best-practices. In short, we train a domain classifier on the <a href="https://api.semanticscholar.org/corpus">semantic scholar dataset</a> by taking full advantage of <code>great-ai</code>. Subsequently, we create a production-ready deployment.</p>
|
||||
<h1 id="simple-example-data-engineering">Simple example: data engineering<a class="anchor-link" href="#simple-example-data-engineering">¶</a></h1><p>Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best practices. In short, we train a domain classifier on the <a href="https://api.semanticscholar.org/corpus">semantic scholar dataset</a> by taking full advantage of <code>great-ai</code>. Subsequently, we create a production-ready deployment.</p>
|
||||
<p><img src="/media/scope-data.svg" alt="position of this step in the lifecycle"></p>
|
||||
<blockquote><p>The blue boxes show the steps of a typical AI-development lifecycle implemented in this notebook.</p>
|
||||
</blockquote>
|
||||
|
|
@ -2416,7 +2416,7 @@ add_ground_truth(X, y, train_split_ratio=0.8, test_split_ratio=0.2)</div>
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 16, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ In order to simplify your training data management, `great-ai` provide two compl
|
|||
At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call [great_ai.add_ground_truth][]. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, [great_ai.query_ground_truth][] can be called to get a filtered view of the training data.
|
||||
|
||||
!!! important "Train-test-validation splits"
|
||||
It is a best-practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.
|
||||
It is a best practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.
|
||||
|
||||
With [great_ai.add_ground_truth][], there is an option to tag the samples with `train`, `test`, and `validation` randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
@ -912,7 +912,7 @@
|
|||
<p>At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, <a class="autorefs autorefs-internal" href="../../reference/#great_ai.query_ground_truth">great_ai.query_ground_truth</a> can be called to get a filtered view of the training data.</p>
|
||||
<div class="admonition important">
|
||||
<p class="admonition-title">Train-test-validation splits</p>
|
||||
<p>It is a best-practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.</p>
|
||||
<p>It is a best practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.</p>
|
||||
<p>With <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>, there is an option to tag the samples with <code>train</code>, <code>test</code>, and <code>validation</code> randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.</p>
|
||||
</div>
|
||||
<p>The nice thing about this is that the 'input-expected output' pairs are stored as traces. Thus, they behave exactly like regular prediction traces.</p>
|
||||
|
|
@ -968,7 +968,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
@ -1080,7 +1080,7 @@
|
|||
<p>Lastly, all version of the remote object can be deleted by calling <code>LargeFileS3("my-file").delete()</code>. It will still reside in your local cache afterwards; its deletion will happen next time your local cache has to be pruned.</p>
|
||||
</details>
|
||||
<h2 id="from-the-command-line">From the command-line<a class="headerlink" href="#from-the-command-line" title="Permanent link">#</a></h2>
|
||||
<p>The main reason for using the <code>large-file</code> or <code>python3 -m great_ai.large_file</code> commands is to upload or download models from the terminal. For example, when building a docker image, it is best-practice to cache the referred models.</p>
|
||||
<p>The main reason for using the <code>large-file</code> or <code>python3 -m great_ai.large_file</code> commands is to upload or download models from the terminal. For example, when building a docker image, it is best practice to cache the referred models.</p>
|
||||
<h3 id="setup">Setup<a class="headerlink" href="#setup" title="Permanent link">#</a></h3>
|
||||
<p>Create an .ini file (or use <em>~/.aws/credentials</em>). It may look like this:</p>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="na">aws_region_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">your_region_like_eu-west-2</span><span class="w"></span>
|
||||
|
|
@ -1117,7 +1117,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 16, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -70,7 +70,7 @@ Oftentimes, especially when working with data-heavy applications, large files ca
|
|||
|
||||
## From the command-line
|
||||
|
||||
The main reason for using the `large-file` or `python3 -m great_ai.large_file` commands is to upload or download models from the terminal. For example, when building a docker image, it is best-practice to cache the referred models.
|
||||
The main reason for using the `large-file` or `python3 -m great_ai.large_file` commands is to upload or download models from the terminal. For example, when building a docker image, it is best practice to cache the referred models.
|
||||
|
||||
### Setup
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
13
index.html
13
index.html
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
@ -964,10 +964,11 @@
|
|||
<ul class="task-list">
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save prediction traces of each prediction including arguments and model versions</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save feedback and merge it into a ground-truth database</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Version and store models and data on shared infrastructure <em>(MongoDB GridFS, S3-compatible storage, shared local-volume)</em></li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Version and store models and data on shared infrastructure <em>(MongoDB GridFS, S3-compatible storage, shared volume)</em></li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Input validation</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Sensible cache-policy</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Graceful error handling</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Seamless support for both synchronous and asynchronous inference methods</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Easy integration with remote GreatAI instances</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing</li>
|
||||
|
|
@ -977,7 +978,7 @@
|
|||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Auto-reload for development</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Docker support for deployment</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Deployable Jupyter Notebooks</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Dashboard for high-level overview and analysing traces</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Dashboard for online monitoring and analysing traces</li>
|
||||
</ul>
|
||||
<h2 id="roadmap">Roadmap<a class="headerlink" href="#roadmap" title="Permanent link">#</a></h2>
|
||||
<ul class="task-list">
|
||||
|
|
@ -1020,7 +1021,7 @@
|
|||
<p><img alt="demo screen capture" loading="lazy" src="media/demo.gif" /></p>
|
||||
<div class="admonition success">
|
||||
<p class="admonition-title">Success</p>
|
||||
<p>Your GreatAI service is ready for production use. Many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best-practices</a> are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.</p>
|
||||
<p>Your GreatAI service is ready for production use. Many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best practices</a> are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.</p>
|
||||
</div>
|
||||
<h2 id="why-is-this-great">Why is this GREAT?<a class="headerlink" href="#why-is-this-great" title="Permanent link">#</a></h2>
|
||||
<p><img alt="scope of GreatAI" src="media/scope-simple.drawio.svg" /></p>
|
||||
|
|
@ -1034,7 +1035,7 @@
|
|||
</ul>
|
||||
<h2 id="why-greatai">Why GreatAI?<a class="headerlink" href="#why-greatai" title="Permanent link">#</a></h2>
|
||||
<p>There are other, existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity to use them, do that because they're great.</p>
|
||||
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. <mark>GreatAI is designed to be as simple to use as possible.</mark> Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best-practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
|
||||
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. <mark>GreatAI is designed to be as simple to use as possible.</mark> Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
|
||||
<div style="display: flex; justify-content: space-evenly; flex-wrap: wrap;">
|
||||
<p><a class="md-button md-button--primary" href="https://pypi.org/project/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg></span> Find it on PyPI</a></p>
|
||||
<p><a class="md-button md-button--primary" href="https://hub.docker.com/repository/docker/schmelczera/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M349.9 236.3h-66.1v-59.4h66.1v59.4zm0-204.3h-66.1v60.7h66.1V32zm78.2 144.8H362v59.4h66.1v-59.4zm-156.3-72.1h-66.1v60.1h66.1v-60.1zm78.1 0h-66.1v60.1h66.1v-60.1zm276.8 100c-14.4-9.7-47.6-13.2-73.1-8.4-3.3-24-16.7-44.9-41.1-63.7l-14-9.3-9.3 14c-18.4 27.8-23.4 73.6-3.7 103.8-8.7 4.7-25.8 11.1-48.4 10.7H2.4c-8.7 50.8 5.8 116.8 44 162.1 37.1 43.9 92.7 66.2 165.4 66.2 157.4 0 273.9-72.5 328.4-204.2 21.4.4 67.6.1 91.3-45.2 1.5-2.5 6.6-13.2 8.5-17.1l-13.3-8.9zm-511.1-27.9h-66v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm-78.1-72.1h-66.1v60.1h66.1v-60.1z"/></svg></span> Find it on DockerHub</a></p>
|
||||
|
|
@ -1050,7 +1051,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
9
index.md
9
index.md
|
|
@ -26,10 +26,11 @@ Applying AI is becoming increasingly easier but many case studies have shown tha
|
|||
|
||||
- [x] Save prediction traces of each prediction including arguments and model versions
|
||||
- [x] Save feedback and merge it into a ground-truth database
|
||||
- [x] Version and store models and data on shared infrastructure *(MongoDB GridFS, S3-compatible storage, shared local-volume)*
|
||||
- [x] Version and store models and data on shared infrastructure *(MongoDB GridFS, S3-compatible storage, shared volume)*
|
||||
- [x] Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration
|
||||
- [x] Input validation
|
||||
- [x] Sensible cache-policy
|
||||
- [x] Graceful error handling
|
||||
- [x] Seamless support for both synchronous and asynchronous inference methods
|
||||
- [x] Easy integration with remote GreatAI instances
|
||||
- [x] Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing
|
||||
|
|
@ -39,7 +40,7 @@ Applying AI is becoming increasingly easier but many case studies have shown tha
|
|||
- [x] Auto-reload for development
|
||||
- [x] Docker support for deployment
|
||||
- [x] Deployable Jupyter Notebooks
|
||||
- [x] Dashboard for high-level overview and analysing traces
|
||||
- [x] Dashboard for online monitoring and analysing traces
|
||||
|
||||
## Roadmap
|
||||
|
||||
|
|
@ -80,7 +81,7 @@ great-ai demo.py
|
|||
{ loading=lazy }
|
||||
|
||||
!!! success
|
||||
Your GreatAI service is ready for production use. Many of the [SE4ML best-practices](https://se-ml.github.io){ target=_blank } are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.
|
||||
Your GreatAI service is ready for production use. Many of the [SE4ML best practices](https://se-ml.github.io){ target=_blank } are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.
|
||||
|
||||
## Why is this GREAT?
|
||||
|
||||
|
|
@ -98,7 +99,7 @@ GreatAI fits between the prototype and deployment phases of your (or your organi
|
|||
|
||||
There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity to use them, do that because they're great.
|
||||
|
||||
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. ==GreatAI is designed to be as simple to use as possible.== Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best-practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
|
||||
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. ==GreatAI is designed to be as simple to use as possible.== Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
|
||||
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly; flex-wrap: wrap;" markdown>
|
||||
|
|
|
|||
|
|
@ -187,7 +187,7 @@
|
|||
<div data-drawio-colors="color: #ffffff; " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(255, 255, 255); line-height: 1.2; pointer-events: none; white-space: normal; overflow-wrap: normal;">
|
||||
<font style="font-size: 14px">
|
||||
Artifact combining the model with deplyoment best-practices
|
||||
Artifact combining the model with deplyoment best practices
|
||||
</font>
|
||||
</div>
|
||||
</div>
|
||||
|
|
|
|||
|
Before Width: | Height: | Size: 19 KiB After Width: | Height: | Size: 19 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
@ -1252,7 +1252,7 @@
|
|||
Bases: <code><span title="typing.Generic">Generic</span>[<span title="great_ai.deploy.great_ai.T">T</span>, <span title="great_ai.deploy.great_ai.V">V</span>]</code></p>
|
||||
|
||||
|
||||
<p>Wrapper for a prediction function providing the implementation of SE4ML best-practices.</p>
|
||||
<p>Wrapper for a prediction function providing the implementation of SE4ML best practices.</p>
|
||||
<p>Provides caching (with argument freezing), a TracingContext during execution, the
|
||||
scaffolding of HTTP endpoints using FastAPI and a dashboard using Dash.</p>
|
||||
<p>IMPORTANT: when a request is served from cache, no new trace is created. Thus, the
|
||||
|
|
@ -1568,7 +1568,7 @@ registered through use_model.</p></td>
|
|||
<span class="normal"><a href="#__codelineno-0-319">319</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-320">320</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-321">321</a></span></pre></div></td><td class="code"><div><pre><span></span><code><a id="__codelineno-0-44" name="__codelineno-0-44"></a><span class="k">class</span> <span class="nc">GreatAI</span><span class="p">(</span><span class="n">Generic</span><span class="p">[</span><span class="n">T</span><span class="p">,</span> <span class="n">V</span><span class="p">]):</span>
|
||||
<a id="__codelineno-0-45" name="__codelineno-0-45"></a> <span class="sd">"""Wrapper for a prediction function providing the implementation of SE4ML best-practices.</span>
|
||||
<a id="__codelineno-0-45" name="__codelineno-0-45"></a> <span class="sd">"""Wrapper for a prediction function providing the implementation of SE4ML best practices.</span>
|
||||
<a id="__codelineno-0-46" name="__codelineno-0-46"></a>
|
||||
<a id="__codelineno-0-47" name="__codelineno-0-47"></a><span class="sd"> Provides caching (with argument freezing), a TracingContext during execution, the</span>
|
||||
<a id="__codelineno-0-48" name="__codelineno-0-48"></a><span class="sd"> scaffolding of HTTP endpoints using FastAPI and a dashboard using Dash.</span>
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
@ -2474,7 +2474,15 @@ language tagging.</p>
|
|||
<span class="normal"><a href="#__codelineno-0-109">109</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-110">110</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-111">111</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-112">112</a></span></pre></div></td><td class="code"><div><pre><span></span><code><a id="__codelineno-0-15" name="__codelineno-0-15"></a><span class="k">def</span> <span class="nf">evaluate_ranking</span><span class="p">(</span>
|
||||
<span class="normal"><a href="#__codelineno-0-112">112</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-113">113</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-114">114</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-115">115</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-116">116</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-117">117</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-118">118</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-119">119</a></span>
|
||||
<span class="normal"><a href="#__codelineno-0-120">120</a></span></pre></div></td><td class="code"><div><pre><span></span><code><a id="__codelineno-0-15" name="__codelineno-0-15"></a><span class="k">def</span> <span class="nf">evaluate_ranking</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-16" name="__codelineno-0-16"></a> <span class="n">expected</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">T</span><span class="p">],</span>
|
||||
<a id="__codelineno-0-17" name="__codelineno-0-17"></a> <span class="n">actual_scores</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="nb">float</span><span class="p">],</span>
|
||||
<a id="__codelineno-0-18" name="__codelineno-0-18"></a> <span class="n">target_recall</span><span class="p">:</span> <span class="nb">float</span><span class="p">,</span>
|
||||
|
|
@ -2529,49 +2537,57 @@ language tagging.</p>
|
|||
<a id="__codelineno-0-67" name="__codelineno-0-67"></a> <span class="p">(</span><span class="n">v</span> <span class="o"><</span> <span class="n">classes</span><span class="p">[</span><span class="n">i</span><span class="p">])</span> <span class="k">if</span> <span class="n">reverse_order</span> <span class="k">else</span> <span class="p">(</span><span class="n">v</span> <span class="o">></span> <span class="n">classes</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
|
||||
<a id="__codelineno-0-68" name="__codelineno-0-68"></a> <span class="k">for</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">expected</span>
|
||||
<a id="__codelineno-0-69" name="__codelineno-0-69"></a> <span class="p">]</span>
|
||||
<a id="__codelineno-0-70" name="__codelineno-0-70"></a> <span class="n">precision</span><span class="p">,</span> <span class="n">recall</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="n">precision_recall_curve</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-71" name="__codelineno-0-71"></a> <span class="n">binarized_expected</span><span class="p">,</span> <span class="n">actual_scores</span>
|
||||
<a id="__codelineno-0-72" name="__codelineno-0-72"></a> <span class="p">)</span>
|
||||
<a id="__codelineno-0-73" name="__codelineno-0-73"></a>
|
||||
<a id="__codelineno-0-74" name="__codelineno-0-74"></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">disable_interpolation</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-75" name="__codelineno-0-75"></a> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">precision</span><span class="p">)):</span>
|
||||
<a id="__codelineno-0-76" name="__codelineno-0-76"></a> <span class="n">precision</span><span class="p">[</span><span class="n">j</span><span class="p">]</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">precision</span><span class="p">[</span><span class="n">j</span> <span class="o">-</span> <span class="mi">1</span><span class="p">],</span> <span class="n">precision</span><span class="p">[</span><span class="n">j</span><span class="p">])</span>
|
||||
<a id="__codelineno-0-77" name="__codelineno-0-77"></a>
|
||||
<a id="__codelineno-0-78" name="__codelineno-0-78"></a> <span class="n">closest_recall_index</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">argmin</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">abs</span><span class="p">(</span><span class="n">recall</span> <span class="o">-</span> <span class="n">target_recall</span><span class="p">))</span>
|
||||
<a id="__codelineno-0-79" name="__codelineno-0-79"></a> <span class="n">precision_at_closest_recall</span> <span class="o">=</span> <span class="n">precision</span><span class="p">[</span><span class="n">closest_recall_index</span><span class="p">]</span>
|
||||
<a id="__codelineno-0-80" name="__codelineno-0-80"></a> <span class="n">average_precision</span> <span class="o">=</span> <span class="n">average_precision_score</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-81" name="__codelineno-0-81"></a> <span class="n">binarized_expected</span><span class="p">,</span> <span class="n">actual_scores</span>
|
||||
<a id="__codelineno-0-82" name="__codelineno-0-82"></a> <span class="p">)</span>
|
||||
<a id="__codelineno-0-83" name="__codelineno-0-83"></a> <span class="n">results</span><span class="p">[</span><span class="n">classes</span><span class="p">[</span><span class="n">i</span><span class="p">]]</span> <span class="o">=</span> <span class="n">precision_at_closest_recall</span>
|
||||
<a id="__codelineno-0-84" name="__codelineno-0-84"></a>
|
||||
<a id="__codelineno-0-85" name="__codelineno-0-85"></a> <span class="k">if</span> <span class="n">plot</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-86" name="__codelineno-0-86"></a> <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-87" name="__codelineno-0-87"></a> <span class="n">recall</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-88" name="__codelineno-0-88"></a> <span class="n">precision</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-89" name="__codelineno-0-89"></a> <span class="n">label</span><span class="o">=</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="s1">'|'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">[:</span><span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">])</span><span class="si">}</span><span class="s2"> ↔ </span><span class="si">{</span><span class="s1">'|'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">[</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">:])</span><span class="si">}</span><span class="s2"> (P@</span><span class="si">{</span><span class="n">target_recall</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">=</span><span class="si">{</span><span class="n">precision_at_closest_recall</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">, AP=</span><span class="si">{</span><span class="n">average_precision</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">)"</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-70" name="__codelineno-0-70"></a>
|
||||
<a id="__codelineno-0-71" name="__codelineno-0-71"></a> <span class="n">sorted_expected_actual</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-72" name="__codelineno-0-72"></a> <span class="nb">zip</span><span class="p">(</span><span class="n">binarized_expected</span><span class="p">,</span> <span class="n">actual_scores</span><span class="p">),</span> <span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">v</span><span class="p">:</span> <span class="n">v</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">True</span>
|
||||
<a id="__codelineno-0-73" name="__codelineno-0-73"></a> <span class="p">)</span>
|
||||
<a id="__codelineno-0-74" name="__codelineno-0-74"></a> <span class="n">precision</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<a id="__codelineno-0-75" name="__codelineno-0-75"></a> <span class="n">recall</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<a id="__codelineno-0-76" name="__codelineno-0-76"></a> <span class="n">correct</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
<a id="__codelineno-0-77" name="__codelineno-0-77"></a> <span class="k">for</span> <span class="nb">all</span><span class="p">,</span> <span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">score</span><span class="p">)</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">sorted_expected_actual</span><span class="p">,</span> <span class="n">start</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
|
||||
<a id="__codelineno-0-78" name="__codelineno-0-78"></a> <span class="n">correct</span> <span class="o">+=</span> <span class="nb">int</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-79" name="__codelineno-0-79"></a> <span class="n">precision</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">correct</span> <span class="o">/</span> <span class="nb">all</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-80" name="__codelineno-0-80"></a> <span class="n">recall</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="nb">all</span> <span class="o">/</span> <span class="nb">len</span><span class="p">(</span><span class="n">sorted_expected_actual</span><span class="p">))</span>
|
||||
<a id="__codelineno-0-81" name="__codelineno-0-81"></a>
|
||||
<a id="__codelineno-0-82" name="__codelineno-0-82"></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">disable_interpolation</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-83" name="__codelineno-0-83"></a> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">precision</span><span class="p">)</span> <span class="o">-</span> <span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">):</span>
|
||||
<a id="__codelineno-0-84" name="__codelineno-0-84"></a> <span class="n">precision</span><span class="p">[</span><span class="n">j</span><span class="p">]</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">precision</span><span class="p">[</span><span class="n">j</span><span class="p">],</span> <span class="n">precision</span><span class="p">[</span><span class="n">j</span> <span class="o">+</span> <span class="mi">1</span><span class="p">])</span>
|
||||
<a id="__codelineno-0-85" name="__codelineno-0-85"></a>
|
||||
<a id="__codelineno-0-86" name="__codelineno-0-86"></a> <span class="n">closest_recall_index</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">argmin</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">abs</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">recall</span><span class="p">)</span> <span class="o">-</span> <span class="n">target_recall</span><span class="p">))</span>
|
||||
<a id="__codelineno-0-87" name="__codelineno-0-87"></a> <span class="n">precision_at_closest_recall</span> <span class="o">=</span> <span class="n">precision</span><span class="p">[</span><span class="n">closest_recall_index</span><span class="p">]</span>
|
||||
<a id="__codelineno-0-88" name="__codelineno-0-88"></a> <span class="n">average_precision</span> <span class="o">=</span> <span class="n">average_precision_score</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-89" name="__codelineno-0-89"></a> <span class="n">binarized_expected</span><span class="p">,</span> <span class="n">actual_scores</span>
|
||||
<a id="__codelineno-0-90" name="__codelineno-0-90"></a> <span class="p">)</span>
|
||||
<a id="__codelineno-0-91" name="__codelineno-0-91"></a>
|
||||
<a id="__codelineno-0-92" name="__codelineno-0-92"></a> <span class="k">if</span> <span class="n">plot</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-93" name="__codelineno-0-93"></a> <span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="s2">"upper right"</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-94" name="__codelineno-0-94"></a> <span class="n">ax</span><span class="o">.</span><span class="n">axvline</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">target_recall</span><span class="p">,</span> <span class="n">linestyle</span><span class="o">=</span><span class="s2">"--"</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="s2">"#55c6bb"</span><span class="p">,</span> <span class="n">linewidth</span><span class="o">=</span><span class="mf">2.0</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-95" name="__codelineno-0-95"></a>
|
||||
<a id="__codelineno-0-96" name="__codelineno-0-96"></a> <span class="k">if</span> <span class="n">title</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-97" name="__codelineno-0-97"></a> <span class="n">title</span> <span class="o">=</span> <span class="s2">"Ranking evaluation"</span>
|
||||
<a id="__codelineno-0-98" name="__codelineno-0-98"></a>
|
||||
<a id="__codelineno-0-99" name="__codelineno-0-99"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="sa">f</span><span class="s1">'</span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="s1"> (</span><span class="si">{</span><span class="s2">" < "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">)</span><span class="si">}</span><span class="s1">)'</span><span class="p">,</span> <span class="n">pad</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-100" name="__codelineno-0-100"></a>
|
||||
<a id="__codelineno-0-101" name="__codelineno-0-101"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s2">"Recall"</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-102" name="__codelineno-0-102"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_ylabel</span><span class="p">(</span><span class="s2">"Precision"</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-91" name="__codelineno-0-91"></a> <span class="n">results</span><span class="p">[</span><span class="n">classes</span><span class="p">[</span><span class="n">i</span><span class="p">]]</span> <span class="o">=</span> <span class="n">precision_at_closest_recall</span>
|
||||
<a id="__codelineno-0-92" name="__codelineno-0-92"></a>
|
||||
<a id="__codelineno-0-93" name="__codelineno-0-93"></a> <span class="k">if</span> <span class="n">plot</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-94" name="__codelineno-0-94"></a> <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-95" name="__codelineno-0-95"></a> <span class="n">recall</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-96" name="__codelineno-0-96"></a> <span class="n">precision</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-97" name="__codelineno-0-97"></a> <span class="n">label</span><span class="o">=</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="s1">'|'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">[:</span><span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">])</span><span class="si">}</span><span class="s2"> ↔ </span><span class="si">{</span><span class="s1">'|'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">[</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">:])</span><span class="si">}</span><span class="s2"> (P@</span><span class="si">{</span><span class="n">target_recall</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">=</span><span class="si">{</span><span class="n">precision_at_closest_recall</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">, AP=</span><span class="si">{</span><span class="n">average_precision</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">)"</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-98" name="__codelineno-0-98"></a> <span class="p">)</span>
|
||||
<a id="__codelineno-0-99" name="__codelineno-0-99"></a>
|
||||
<a id="__codelineno-0-100" name="__codelineno-0-100"></a> <span class="k">if</span> <span class="n">plot</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-101" name="__codelineno-0-101"></a> <span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="s2">"upper right"</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-102" name="__codelineno-0-102"></a> <span class="n">ax</span><span class="o">.</span><span class="n">axvline</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">target_recall</span><span class="p">,</span> <span class="n">linestyle</span><span class="o">=</span><span class="s2">"--"</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="s2">"#55c6bb"</span><span class="p">,</span> <span class="n">linewidth</span><span class="o">=</span><span class="mf">2.0</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-103" name="__codelineno-0-103"></a>
|
||||
<a id="__codelineno-0-104" name="__codelineno-0-104"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_xticks</span><span class="p">([</span><span class="n">target_recall</span><span class="p">]</span> <span class="o">+</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">ax</span><span class="o">.</span><span class="n">get_xticks</span><span class="p">()))</span>
|
||||
<a id="__codelineno-0-105" name="__codelineno-0-105"></a>
|
||||
<a id="__codelineno-0-106" name="__codelineno-0-106"></a> <span class="k">if</span> <span class="n">plot</span> <span class="ow">and</span> <span class="n">output_svg</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-107" name="__codelineno-0-107"></a> <span class="k">if</span> <span class="n">axes</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-108" name="__codelineno-0-108"></a> <span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
|
||||
<a id="__codelineno-0-109" name="__codelineno-0-109"></a> <span class="k">elif</span> <span class="n">output_svg</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-110" name="__codelineno-0-110"></a> <span class="n">plt</span><span class="o">.</span><span class="n">savefig</span><span class="p">(</span><span class="n">output_svg</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="s2">"svg"</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-104" name="__codelineno-0-104"></a> <span class="k">if</span> <span class="n">title</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-105" name="__codelineno-0-105"></a> <span class="n">title</span> <span class="o">=</span> <span class="s2">"Ranking evaluation"</span>
|
||||
<a id="__codelineno-0-106" name="__codelineno-0-106"></a>
|
||||
<a id="__codelineno-0-107" name="__codelineno-0-107"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="sa">f</span><span class="s1">'</span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="s1"> (</span><span class="si">{</span><span class="s2">" < "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">)</span><span class="si">}</span><span class="s1">)'</span><span class="p">,</span> <span class="n">pad</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-108" name="__codelineno-0-108"></a>
|
||||
<a id="__codelineno-0-109" name="__codelineno-0-109"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s2">"Recall"</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-110" name="__codelineno-0-110"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_ylabel</span><span class="p">(</span><span class="s2">"Precision"</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-111" name="__codelineno-0-111"></a>
|
||||
<a id="__codelineno-0-112" name="__codelineno-0-112"></a> <span class="k">return</span> <span class="n">results</span>
|
||||
<a id="__codelineno-0-112" name="__codelineno-0-112"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_xticks</span><span class="p">([</span><span class="n">target_recall</span><span class="p">]</span> <span class="o">+</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">ax</span><span class="o">.</span><span class="n">get_xticks</span><span class="p">()))</span>
|
||||
<a id="__codelineno-0-113" name="__codelineno-0-113"></a>
|
||||
<a id="__codelineno-0-114" name="__codelineno-0-114"></a> <span class="k">if</span> <span class="n">plot</span> <span class="ow">and</span> <span class="n">output_svg</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-115" name="__codelineno-0-115"></a> <span class="k">if</span> <span class="n">axes</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-116" name="__codelineno-0-116"></a> <span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
|
||||
<a id="__codelineno-0-117" name="__codelineno-0-117"></a> <span class="k">elif</span> <span class="n">output_svg</span><span class="p">:</span>
|
||||
<a id="__codelineno-0-118" name="__codelineno-0-118"></a> <span class="n">plt</span><span class="o">.</span><span class="n">savefig</span><span class="p">(</span><span class="n">output_svg</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="s2">"svg"</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-119" name="__codelineno-0-119"></a>
|
||||
<a id="__codelineno-0-120" name="__codelineno-0-120"></a> <span class="k">return</span> <span class="n">results</span>
|
||||
</code></pre></div></td></tr></table></div>
|
||||
</details>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
52
sitemap.xml
52
sitemap.xml
|
|
@ -2,132 +2,132 @@
|
|||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-17</lastmod>
|
||||
<lastmod>2022-07-24</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
</urlset>
|
||||
BIN
sitemap.xml.gz
BIN
sitemap.xml.gz
Binary file not shown.
|
|
@ -40,7 +40,7 @@ SageMaker offers the most comprehensive suite of tools and service; most importa
|
|||
|
||||
When it comes to open-source libraries, we can find the MLOps libraries of both TensorFlow and PyTorch: TensorFlow Extended (TFX) \cite{baylor2017tfx} and TorchX\footnote{\href{https://pytorch.org/torchx/latest/}{pytorch.org/torchx/latest}}. TFX comes with a more mature set of features with the caveat that initial time-investment is needed for their setup. The features of TorchX only concern the distributed deployment to a wide range of providers, including Kubernetes (K8s), AWS Batch, or Ray \cite{moritz2018ray}. There is no augmentation for the SE4ML best practices. Given the tight coupling between these libraries and their corresponding ML frameworks, they cannot generalise to models\footnote{The Open Neural Network Exchange (\href{https://onnx.ai/}{onnx.ai}) format could be an option for overcoming these incompatibilities, however, a more universal support is needed for seamless integration.} or algorithms of other frameworks and technologies.
|
||||
|
||||
Open-source platforms also exist such as MLflow and Seldon Core. They both rely on Kubernetes to provide their features. MLflow puts more emphasis on the training phase (in deployment, it lacks a feedback loop which is essential for reaching many of the best-practices), while Seldon Core focuses on the deployment stage. The latter comes integrated with a powerful explanation engine, Alibi Explain \cite{klaise2021alibi}. It also boasts the most comprehensive suite of features including outlier detection, online model selection (with multi-armed bandit theory), and distributed tracing. In short, it seems to be the ideal candidate for the title of \textit{framework for robust end-to-end AI deployments}. Its only downside is the amount of complexity propagated to its clients: it is built on top of Kubernetes, and relies on Helm, Ambasador/Istio, Prometheus, and Jaeger for its features. Hence, the first step in using it is setting up a K8s cluster with all the required components, then when it comes to model deployment, a Kubernetes configuration file has to be created to make use of Seldon's Custom Resource Definition. These are smaller obstacles if the project is already built on top of K8s; however, even then, software engineers with strong cloud and DevOps background are actively required for using Seldon Core.
|
||||
Open-source platforms also exist such as MLflow and Seldon Core. They both rely on Kubernetes to provide their features. MLflow puts more emphasis on the training phase (in deployment, it lacks a feedback loop which is essential for reaching many of the best practices), while Seldon Core focuses on the deployment stage. The latter comes integrated with a powerful explanation engine, Alibi Explain \cite{klaise2021alibi}. It also boasts the most comprehensive suite of features including outlier detection, online model selection (with multi-armed bandit theory), and distributed tracing. In short, it seems to be the ideal candidate for the title of \textit{framework for robust end-to-end AI deployments}. Its only downside is the amount of complexity propagated to its clients: it is built on top of Kubernetes, and relies on Helm, Ambasador/Istio, Prometheus, and Jaeger for its features. Hence, the first step in using it is setting up a K8s cluster with all the required components, then when it comes to model deployment, a Kubernetes configuration file has to be created to make use of Seldon's Custom Resource Definition. These are smaller obstacles if the project is already built on top of K8s; however, even then, software engineers with strong cloud and DevOps background are actively required for using Seldon Core.
|
||||
|
||||
Additionally, increasing attention is given to ML deployments in embedded systems both from a theoretical \cite{john2020ai} and practical \cite{prado2020bonseyes} point of view. Prado et al. \cite{prado2020bonseyes} survey the available deployment frameworks and end-to-end solutions including those for embedded devices. They note their inefficiencies that come from the lack of features and too much rigidity. They introduce their framework for embedded AI deployments which can be used out-of-the box but also lets the users easily replace and extend its pipeline with steps to fit their changing needs and advancements of the field. While Meenu et al. \cite{john2020ai} present and compare different architectural choices for large-scale deployments in edge-computing. They also note that: \textit{"...there is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature"}.
|
||||
|
||||
|
|
@ -68,7 +68,7 @@ No DevOps dependencies\textsuperscript{4}& & &
|
|||
\begin{tablenotes}
|
||||
\item[1] For privacy and accountability reasons. \cite{bosch2021engineering}
|
||||
\item[2] Minimising required glue code. \cite{sculley2015hidden}
|
||||
\item[3] Implementing best-practices. \cite{serban2020adoption,serban2021practices}
|
||||
\item[3] Implementing best practices. \cite{serban2020adoption,serban2021practices}
|
||||
\item[4] Easy integration into existing processes. \cite{haakman2021ai,thiee2021systematic}
|
||||
\item[*] Only partial support.
|
||||
\end{tablenotes}
|
||||
|
|
|
|||
|
|
@ -41,7 +41,7 @@ SSC is much larger: it contains over 80 million abstracts. Having more data cert
|
|||
MAG needs no further preprocessing if we aim to match SciBERT's setup \cite{beltagy2019scibert}. But since SSC contains a heap of metadata, the relevant parts have to be extracted and preprocessed. In this case, these are the concatenation of the abstract's text, paper's title and the journal's name along with the paper's domains (there can be multiple domains for a single paper, it is a multi-label classification task). Lastly, the non-English entries are discarded because we only expect to process papers in English.
|
||||
|
||||
\begin{displayquote}
|
||||
\textbf{How should we preprocess the data?} These simple processing steps (filter, map, project) are almost always present in the data science life-cycle. For example, cleaning the input text from various HTML, OCR, PDF, or \LaTeX \hskip 0.12cm extraction artifacts is almost always necessary for text analysis. This is captured in the AI best-practices collection under the following category: \textit{Write Reusable Scripts for Data Cleaning and Merging}. Also, the best practice of \textit{Test all Feature Extraction Code} is somewhat applicable: the applied processing steps must not introduce unwanted artifacts.
|
||||
\textbf{How should we preprocess the data?} These simple processing steps (filter, map, project) are almost always present in the data science life-cycle. For example, cleaning the input text from various HTML, OCR, PDF, or \LaTeX \hskip 0.12cm extraction artifacts is almost always necessary for text analysis. This is captured in the AI best practices collection under the following category: \textit{Write Reusable Scripts for Data Cleaning and Merging}. Also, the best practice of \textit{Test all Feature Extraction Code} is somewhat applicable: the applied processing steps must not introduce unwanted artifacts.
|
||||
\end{displayquote}
|
||||
|
||||
\subsection{Methods}
|
||||
|
|
@ -50,7 +50,7 @@ Our aims are twofold: (1) to evaluate a sentence classification model on MAG and
|
|||
|
||||
It seems reasonable that only considering the distribution (frequencies) of individual terms may be sufficient. To test this hypothesis, a unigram language model (Multinomial Naïve Bayes) is constructed and its accuracy is compared with SciBERT's. The former definitely aligns with the advice to \textit{Use The Most Efficient Models}\footnote{\href{https://se-ml.github.io/best_practices/02-efficient-models/}{se-ml.github.io/best\_practices/02-efficient-models}}.
|
||||
|
||||
Using the MNB implementation of scikit-learn \cite{pedregosa2011scikit}, it only took a couple of lines to create, hyperparameter-optimise, and test a text classifier. Including data loading and visualisations, it takes 71 lines of code (LOC) to be more precise. \footnote{The code is available at \href{https://great-ai.scoutinscience.com/tutorial/}{great-ai.scoutinscience.com/tutorial}.} This further proves relatively how simple it is to use standard packages. The code can be considered for satisfying the \textit{Automate Hyper-Parameter Optimisation} best-practice, since it also implements an automated hyperparameter sweep.
|
||||
Using the MNB implementation of scikit-learn \cite{pedregosa2011scikit}, it only took a couple of lines to create, hyperparameter-optimise, and test a text classifier. Including data loading and visualisations, it takes 71 lines of code (LOC) to be more precise. \footnote{The code is available at \href{https://great-ai.scoutinscience.com/tutorial/}{great-ai.scoutinscience.com/tutorial}.} This further proves relatively how simple it is to use standard packages. The code can be considered for satisfying the \textit{Automate Hyper-Parameter Optimisation} best practice, since it also implements an automated hyperparameter sweep.
|
||||
|
||||
The sentences are tokenised into words and vectorised with TF-IDF (with logarithmic term frequency) \cite{buckley1985implementation}, the hyperparameters found via 3-fold cross-validation on the \textit{train} split lead to filtering out tokens which occur in fewer than 5 documents or more than 5\% of the documents.
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 1.5 MiB After Width: | Height: | Size: 424 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 208 KiB After Width: | Height: | Size: 170 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 MiB After Width: | Height: | Size: 282 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.7 MiB After Width: | Height: | Size: 1,015 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 792 KiB After Width: | Height: | Size: 408 KiB |
|
|
@ -6,7 +6,7 @@
|
|||
"source": [
|
||||
"# Harden and deploy your app\n",
|
||||
"\n",
|
||||
"Finally, it's time to deploy your model. But before that, you have to make sure you follow AI deployment [best-practices](https://se-ml.github.io/). In the past, this step was too often either the source of unexpected struggle, or worse, simply ignored.\n",
|
||||
"Finally, it's time to deploy your model. But before that, you have to make sure you follow AI deployment [best practices](https://se-ml.github.io/). In the past, this step was too often either the source of unexpected struggle, or worse, simply ignored.\n",
|
||||
"\n",
|
||||
"With `GreatAI`, it has become a matter of 4 lines of code."
|
||||
]
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
@ -1918,7 +1918,7 @@ Licensed under the Apache License, Version 2.0.
|
|||
</div>
|
||||
<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
|
||||
</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput " data-mime-type="text/markdown">
|
||||
<h1 id="harden-and-deploy-your-app">Harden and deploy your app<a class="anchor-link" href="#harden-and-deploy-your-app">¶</a></h1><p>Finally, it's time to deploy your model. But before that, you have to make sure you follow AI deployment <a href="https://se-ml.github.io/">best-practices</a>. In the past, this step was too often either the source of unexpected struggle, or worse, simply ignored.</p>
|
||||
<h1 id="harden-and-deploy-your-app">Harden and deploy your app<a class="anchor-link" href="#harden-and-deploy-your-app">¶</a></h1><p>Finally, it's time to deploy your model. But before that, you have to make sure you follow AI deployment <a href="https://se-ml.github.io/">best practices</a>. In the past, this step was too often either the source of unexpected struggle, or worse, simply ignored.</p>
|
||||
<p>With <code>GreatAI</code>, it has become a matter of 4 lines of code.</p>
|
||||
|
||||
</div>
|
||||
|
|
@ -2197,7 +2197,7 @@ def predict_domain(sentence, model):
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
|
||||
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue