andras/great-ai
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 4 Packages Wiki Activity Actions

Deployed 63a4598 with MkDocs version: 1.3.0

Browse source
This commit is contained in:
2022-07-09 21:12:02 +00:00
parent a388bcf449
commit 38c201a13f
51 changed files with 7876 additions and 1447 deletions
Download patch file Download diff file Expand all files Collapse all files

246
index.html
View file

@ -12,7 +12,7 @@
<meta name="author" content="András Schmelczer">
<link rel="icon" href="assets/images/favicon.png">
<link rel="icon" href="media/favicon.ico">
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
@ -70,7 +70,7 @@
<div data-md-component="skip">
<a href="#overview-of-greatai" class="md-skip">
<a href="#features" class="md-skip">
Skip to content
</a>
@ -241,8 +241,6 @@
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
<label class="md-nav__link md-nav__link--active" for="__toc">
Overview
@ -259,8 +257,6 @@
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
@ -294,6 +290,13 @@
Why GreatAI?
</a>
</li>
<li class="md-nav__item">
<a href="#production-use" class="md-nav__link">
Production use
</a>
</li>
</ul>
@ -311,13 +314,74 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
<label class="md-nav__link" for="__nav_2">
Tutorial
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
<label class="md-nav__title" for="__nav_2">
<span class="md-nav__icon md-icon"></span>
Tutorial
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="tutorials/" class="md-nav__link">
Tutorials
<a href="tutorial/" class="md-nav__link">
Train and deploy a SOTA model
</a>
</li>
<li class="md-nav__item">
<a href="tutorial/train/" class="md-nav__link">
Train your model
</a>
</li>
<li class="md-nav__item">
<a href="tutorial/deploy/" class="md-nav__link">
Harden and deploy your app
</a>
</li>
</ul>
</nav>
</li>
@ -326,13 +390,107 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3">
User Guides
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
User Guides
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
<label class="md-nav__link" for="__nav_3_1">
Simple AI lifecycle with classification
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
<label class="md-nav__title" for="__nav_3_1">
<span class="md-nav__icon md-icon"></span>
Simple AI lifecycle with classification
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="how-to-guides/" class="md-nav__link">
How-To Guides
<a href="simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
@ -364,21 +522,6 @@
<li class="md-nav__item">
<a href="simple-mag/train/" class="md-nav__link">
Train a field of study (domain) classification of sentences on the MAG dataset
</a>
</li>
</ul>
</nav>
</div>
@ -396,8 +539,6 @@
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
@ -431,6 +572,13 @@
Why GreatAI?
</a>
</li>
<li class="md-nav__item">
<a href="#production-use" class="md-nav__link">
Production use
</a>
</li>
</ul>
@ -454,7 +602,11 @@
<h1 id="overview-of-greatai">Overview of GreatAI<a class="headerlink" href="#overview-of-greatai" title="Permanent link">#</a></h1>
<div style="display: flex; justify-content: space-between; align-items: center;">
<h1 style="margin: 0">Overview of GreatAI</h1>
<img src="media/logo.png" width=80>
</div>
<p><a href="https://github.com/schmelczer/great-ai/actions/workflows/check.yml"><img alt="Test" src="https://github.com/schmelczer/great-ai/actions/workflows/test.yml/badge.svg" /></a>
<a href="https://sonar.schmelczer.com/dashboard?id=great-ai"><img alt="Quality Gate Status" src="https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&amp;metric=alert_status" /></a>
<a href="https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml"><img alt="Publish on PyPI" src="https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml/badge.svg" /></a>
@ -464,15 +616,15 @@
<details class="quote">
<summary>Case studies</summary>
<p>"There is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature."
&mdash; <a href="https://ieeexplore.ieee.org/abstract/document/9359253">John et al.</a></p>
<p>"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." &mdash; <a href="https://link.springer.com/article/10.1007/s10664-021-09993-1">Haakman et al.</a></p>
<p>"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." &mdash; <a href="https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf">Sculley et al.</a></p>
<p>"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. &mdash; <a href="https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw">Serban et al.</a></p>
&mdash; <a href="https://ieeexplore.ieee.org/abstract/document/9359253" target="_blank">John et al.</a></p>
<p>"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." &mdash; <a href="https://link.springer.com/article/10.1007/s10664-021-09993-1" target="_blank">Haakman et al.</a></p>
<p>"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." &mdash; <a href="https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf" target="_blank">Sculley et al.</a></p>
<p>"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. &mdash; <a href="https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw" target="_blank">Serban et al.</a></p>
</details>
<h2 id="features">Features<a class="headerlink" href="#features" title="Permanent link">#</a></h2>
<ul class="task-list">
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save prediction traces of each prediction including arguments and model versions</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save feedback and merge it into a ground-truth database</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save feedback and merge it into a ground-truth database <em><img alt="➡" class="twemoji" src="https://twemoji.maxcdn.com/v/latest/svg/27a1.svg" title=":arrow_right:" /> quasi-shadow deployment</em></li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Version and store models and data on shared infrastructure <em>(MongoDB GridFS, S3-compatible storage, shared local-volume)</em></li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Input validation</li>
@ -482,11 +634,11 @@
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Well-tested utilities for common NLP tasks (cleaning, language-tagging, sentence-segmentation, etc.)</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> A simple, unified configuration interface</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Fully-typed API for IntelliSense support</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Fully-typed API for <a href="https://github.com/microsoft/pylance-release" target="_blank">Pylance</a> and <a href="http://mypy-lang.org" target="_blank">MyPy</a> support</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Auto-reload for development</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Deployable Jupyter Notebooks</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Docker support for deployment</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Dashboard for high-level overview and searching traces</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled/><span class="task-list-indicator"></span></label> Shadow deployment</li>
</ul>
<h2 id="hello-world">Hello world<a class="headerlink" href="#hello-world" title="Permanent link">#</a></h2>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a>pip install great-ai
@ -517,7 +669,7 @@
<div class="highlight"><span class="filename">terminal</span><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a>great-ai hello-world.py
</code></pre></div>
<blockquote>
<p>Navigate to <a href="http://127.0.0.1:6060/">localhost:6060</a> in your browser.</p>
<p>Navigate to <a href="http://127.0.0.1:6060">localhost:6060</a> in your browser.</p>
</blockquote>
<div style="display: flex; justify-content: space-evenly;">
<p><img alt="" loading="lazy" src="media/hello-world-dashboard.png" /></p>
@ -525,11 +677,11 @@
</div>
<div class="admonition success">
<p class="admonition-title">Success</p>
<p>Your GreatAI service is ready for production use. Many of the <a href="https://se-ml.github.io/" target="_blank">SE4ML best-practices</a> are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.</p>
<p>Your GreatAI service is ready for production use. Many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best-practices</a> are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.</p>
</div>
<h2 id="why-is-this-great">Why is this GREAT?<a class="headerlink" href="#why-is-this-great" title="Permanent link">#</a></h2>
<p><img alt="scope of GreatAI" src="scope-simple.drawio.svg" /></p>
<p>GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented concerning the following 5 aspects:</p>
<p><img alt="scope of GreatAI" src="media/scope-simple.drawio.svg" /></p>
<p>GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented aiming to achieve the following attributes:</p>
<ul>
<li><strong>G</strong>eneral: use any Python library without restriction</li>
<li><strong>R</strong>obust: have error-handling and well-tested utilities out-of-the-box </li>
@ -538,19 +690,23 @@
<li><strong>T</strong>rustworthy: deploy models that you and society can confidently trust</li>
</ul>
<h2 id="why-greatai">Why GreatAI?<a class="headerlink" href="#why-greatai" title="Permanent link">#</a></h2>
<p>There are other, existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker/" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.</p>
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many <a href="https://se-ml.github.io/" target="_blank">best-practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
<p>There are other, existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.</p>
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many <a href="https://se-ml.github.io" target="_blank">best-practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
<div style="display: flex; justify-content: space-evenly;">
<p><a class="md-button md-button--primary" href="https://pypi.org/project/great-ai/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg></span> Find it on PyPI</a></p>
<p><a class="md-button md-button--primary" href="https://pypi.org/project/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg></span> Find it on PyPI</a></p>
<p><a class="md-button md-button--primary" href="https://hub.docker.com/repository/docker/schmelczera/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M349.9 236.3h-66.1v-59.4h66.1v59.4zm0-204.3h-66.1v60.7h66.1V32zm78.2 144.8H362v59.4h66.1v-59.4zm-156.3-72.1h-66.1v60.1h66.1v-60.1zm78.1 0h-66.1v60.1h66.1v-60.1zm276.8 100c-14.4-9.7-47.6-13.2-73.1-8.4-3.3-24-16.7-44.9-41.1-63.7l-14-9.3-9.3 14c-18.4 27.8-23.4 73.6-3.7 103.8-8.7 4.7-25.8 11.1-48.4 10.7H2.4c-8.7 50.8 5.8 116.8 44 162.1 37.1 43.9 92.7 66.2 165.4 66.2 157.4 0 273.9-72.5 328.4-204.2 21.4.4 67.6.1 91.3-45.2 1.5-2.5 6.6-13.2 8.5-17.1l-13.3-8.9zm-511.1-27.9h-66v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm-78.1-72.1h-66.1v60.1h66.1v-60.1z"/></svg></span> Find it on DockerHub</a></p>
</div>
<h2 id="production-use">Production use<a class="headerlink" href="#production-use" title="Permanent link">#</a></h2>
<p>GreatAI has been battle-tested on the core platform services of <a href="https://www.scoutinscience.com/" target="_blank">ScoutinScience</a>.</p>
<p><img alt="ScoutinScience logo" src="media/scoutinscience.svg#only-light" />
<img alt="ScoutinScience logo" src="media/scoutinscience-white.svg#only-dark" /></p>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
</small>
@ -572,13 +728,13 @@
<a href="tutorials/" class="md-footer__link md-footer__link--next" aria-label="Next: Tutorials" rel="next">
<a href="tutorial/" class="md-footer__link md-footer__link--next" aria-label="Next: Train and deploy a SOTA model" rel="next">
<div class="md-footer__title">
<div class="md-ellipsis">
<span class="md-footer__direction">
Next
</span>
Tutorials
Train and deploy a SOTA model
</div>
</div>
<div class="md-footer__button md-icon">