Deployed 63a4598 with MkDocs version: 1.3.0
This commit is contained in:
parent
a388bcf449
commit
38c201a13f
51 changed files with 7876 additions and 1447 deletions
180
404.html
180
404.html
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="/assets/images/favicon.png">
|
||||
<link rel="icon" href="/media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -244,13 +244,74 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="/tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="/tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="/tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="/tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -259,13 +320,107 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="/how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
<a href="/simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="/simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="/simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -297,21 +452,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="/simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -1,5 +0,0 @@
|
|||
aws_region_name = your_region_like_eu-west-2
|
||||
aws_access_key_id = YOUR_ACCESS_KEY_ID
|
||||
aws_secret_access_key = YOUR_VERY_SECRET_ACCESS_KEY
|
||||
large_files_bucket_name = create_a_bucket_and_put_its_name_here
|
||||
aws_endpoint_url = this is optional, for backblaze, use this: https://s3.us-west-002.backblazeb2.com
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
# Explanation
|
||||
|
||||
A lot more details and discussion about the problem context and approaches of GreatAI can be found in my thesis.
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||
[::fontawesome-solid-graduation-cap: Download](thesis/main.py){ .md-button .md-button--primary }
|
||||
</div>
|
||||
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../assets/images/favicon.png">
|
||||
<link rel="icon" href="../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -69,6 +69,11 @@
|
|||
<label class="md-overlay" for="__drawer"></label>
|
||||
<div data-md-component="skip">
|
||||
|
||||
|
||||
<a href="#explanation" class="md-skip">
|
||||
Skip to content
|
||||
</a>
|
||||
|
||||
</div>
|
||||
<div data-md-component="announce">
|
||||
|
||||
|
|
@ -244,13 +249,74 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -259,13 +325,107 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
<a href="../simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -296,6 +456,8 @@
|
|||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
Explanation
|
||||
|
|
@ -305,21 +467,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -337,6 +484,8 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -357,16 +506,18 @@
|
|||
|
||||
|
||||
|
||||
<h1>Explanation</h1>
|
||||
|
||||
|
||||
<h1 id="explanation">Explanation<a class="headerlink" href="#explanation" title="Permanent link">#</a></h1>
|
||||
<p>A lot more details and discussion about the problem context and approaches of GreatAI can be found in my thesis.</p>
|
||||
<div style="display: flex; justify-content: space-evenly;">
|
||||
<p><a class="md-button md-button--primary" href="thesis/main.py">:<span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M623.1 136.9 340.4 35.7a62.951 62.951 0 0 0-42.43 0L16.05 136.9C6.438 140.4 0 149.6 0 160s6.438 19.65 16.05 23.09l60.02 21.51c-11.89 15.8-20.26 34.16-24.55 53.95C40.05 263.4 32 274.8 32 288c0 9.953 4.814 18.49 11.94 24.36l-24.83 149C17.48 471.1 25 480 34.89 480h58.22c9.887 0 17.41-8.879 15.78-18.63l-24.83-149C91.19 306.5 96 297.1 96 288c0-10.29-5.174-19.03-12.72-24.89a106.301 106.301 0 0 1 24.94-47.03l190.6 68.23a62.951 62.951 0 0 0 42.43 0l282.7-101.2C633.6 179.6 640 170.4 640 160s-6.4-19.6-16.9-23.1zm-272 177.5c-9.4 3.7-20.2 5.6-31.1 5.6a94.767 94.767 0 0 1-32-5.555L142.8 262.5 128 405.3c0 41.3 85.1 74.7 192 74.7 105.1 0 192-33.4 192-74.67l-14.78-142.9L351.1 314.4z"/></svg></span> Download</a></p>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -402,21 +553,6 @@
|
|||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="../simple-mag/train/" class="md-footer__link md-footer__link--next" aria-label="Next: Train a field of study (domain) classification of sentences on the MAG dataset" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
</nav>
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
|
|
|
|||
|
|
@ -1,11 +0,0 @@
|
|||
FROM test
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir ./great_ai
|
||||
|
||||
COPY mongo.ini .
|
||||
COPY deploy.ipynb .
|
||||
|
||||
# RUN python3 -m large_file --backend s3 -secrets ~/.aws/credentials --cache my_first_file.json:3 my_second_file my_folder:0
|
||||
|
||||
CMD ["deploy.ipynb"]
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
# Train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus)
|
||||
|
||||
|
||||
|
||||
## Install
|
||||
|
||||
### System dependencies
|
||||
|
||||
Make sure you have `python3`, `pip`, and `venv` installed.
|
||||
> On Ubuntu, execute: `sudo apt install -y python3 python3-pip python3-venv`
|
||||
|
||||
### Install dependencies
|
||||
```sh
|
||||
python3 -m venv --copies .env
|
||||
source .env/bin/activate
|
||||
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
## Execute
|
||||
|
||||
- [Part 1](src/data.ipynb)
|
||||
- [Part 2](src/train.ipynb)
|
||||
- [Part 3](src/deploy.ipynb)
|
||||
|
|
@ -1,31 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
set -e
|
||||
|
||||
echo "Installing dependencies if necessary"
|
||||
python3 -m pip install --upgrade autoflake isort black[jupyter] mypy flake8
|
||||
|
||||
echo "Formatting and checking $1"
|
||||
|
||||
cd $1
|
||||
|
||||
echo Running autoflake
|
||||
python3 -m autoflake --expand-star-imports --remove-all-unused-imports --ignore-init-module-imports --remove-unused-variables --in-place -r .
|
||||
|
||||
echo Running isort
|
||||
python3 -m isort --profile black --skip .env .
|
||||
|
||||
echo Running black
|
||||
python3 -m black . --exclude .env
|
||||
|
||||
if ls *.py 1> /dev/null 2>&1; then
|
||||
echo Running mypy
|
||||
python3 -m mypy --namespace-packages --ignore-missing-imports --install-types --non-interactive --disallow-untyped-defs --disallow-incomplete-defs --pretty --follow-imports=silent --exclude=external/ --exclude=/build/ .
|
||||
fi
|
||||
|
||||
echo Running Flake8
|
||||
python3 -m flake8 . --count --show-source --statistics --exclude=__init__.py,.env,external --ignore=E501,E722,E402,W503,E203
|
||||
|
||||
cd -
|
||||
|
||||
echo "Finished formatting"
|
||||
|
|
@ -1,471 +0,0 @@
|
|||
|
||||
<!doctype html>
|
||||
<html lang="en" class="no-js">
|
||||
<head>
|
||||
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
|
||||
<meta name="description" content="GreatAI helps you easily transform your prototype AI code into production-ready software.">
|
||||
|
||||
|
||||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../assets/images/favicon.png">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
<title>Train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus) - GreatAI documentation</title>
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
|
||||
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../assets/_mkdocstrings.css">
|
||||
|
||||
<script>__md_scope=new URL("..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="" data-md-color-accent="">
|
||||
|
||||
|
||||
|
||||
<script>var palette=__md_get("__palette");if(palette&&"object"==typeof palette.color)for(var key of Object.keys(palette.color))document.body.setAttribute("data-md-color-"+key,palette.color[key])</script>
|
||||
|
||||
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
|
||||
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
|
||||
<label class="md-overlay" for="__drawer"></label>
|
||||
<div data-md-component="skip">
|
||||
|
||||
|
||||
<a href="#train-a-domain-classifier-on-the-semantic-scholar-dataset" class="md-skip">
|
||||
Skip to content
|
||||
</a>
|
||||
|
||||
</div>
|
||||
<div data-md-component="announce">
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
<header class="md-header" data-md-component="header">
|
||||
<nav class="md-header__inner md-grid" aria-label="Header">
|
||||
<a href=".." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
||||
</a>
|
||||
<label class="md-header__button md-icon" for="__drawer">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
|
||||
</label>
|
||||
<div class="md-header__title" data-md-component="header-title">
|
||||
<div class="md-header__ellipsis">
|
||||
<div class="md-header__topic">
|
||||
<span class="md-ellipsis">
|
||||
GreatAI documentation
|
||||
</span>
|
||||
</div>
|
||||
<div class="md-header__topic" data-md-component="header-topic">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus)
|
||||
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<form class="md-header__option" data-md-component="palette">
|
||||
|
||||
|
||||
|
||||
<input class="md-option" data-md-color-media="" data-md-color-scheme="default" data-md-color-primary="" data-md-color-accent="" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
|
||||
|
||||
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_2" hidden>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12 20 8.69Z"/></svg>
|
||||
</label>
|
||||
|
||||
|
||||
|
||||
|
||||
<input class="md-option" data-md-color-media="" data-md-color-scheme="slate" data-md-color-primary="" data-md-color-accent="" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_2">
|
||||
|
||||
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12c0-2.42-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12 20 8.69Z"/></svg>
|
||||
</label>
|
||||
|
||||
|
||||
</form>
|
||||
|
||||
|
||||
|
||||
<label class="md-header__button md-icon" for="__search">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
|
||||
</label>
|
||||
<div class="md-search" data-md-component="search" role="dialog">
|
||||
<label class="md-search__overlay" for="__search"></label>
|
||||
<div class="md-search__inner" role="search">
|
||||
<form class="md-search__form" name="search">
|
||||
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
|
||||
<label class="md-search__icon md-icon" for="__search">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</label>
|
||||
<nav class="md-search__options" aria-label="Search">
|
||||
|
||||
<button type="reset" class="md-search__icon md-icon" aria-label="Clear" tabindex="-1">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41Z"/></svg>
|
||||
</button>
|
||||
</nav>
|
||||
|
||||
</form>
|
||||
<div class="md-search__output">
|
||||
<div class="md-search__scrollwrap" data-md-scrollfix>
|
||||
<div class="md-search-result" data-md-component="search-result">
|
||||
<div class="md-search-result__meta">
|
||||
Initializing search
|
||||
</div>
|
||||
<ol class="md-search-result__list"></ol>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="md-header__source">
|
||||
<a href="https://github.com/schmelczer/great-ai" title="Go to repository" class="md-source" data-md-component="source">
|
||||
<div class="md-source__icon md-icon">
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
|
||||
</div>
|
||||
<div class="md-source__repository">
|
||||
schmelczer/great-ai
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
</nav>
|
||||
|
||||
</header>
|
||||
|
||||
<div class="md-container" data-md-component="container">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<main class="md-main" data-md-component="main">
|
||||
<div class="md-main__inner md-grid">
|
||||
|
||||
|
||||
|
||||
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
|
||||
<div class="md-sidebar__scrollwrap">
|
||||
<div class="md-sidebar__inner">
|
||||
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
|
||||
<label class="md-nav__title" for="__drawer">
|
||||
<a href=".." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
||||
</a>
|
||||
GreatAI documentation
|
||||
</label>
|
||||
|
||||
<div class="md-nav__source">
|
||||
<a href="https://github.com/schmelczer/great-ai" title="Go to repository" class="md-source" data-md-component="source">
|
||||
<div class="md-source__icon md-icon">
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
|
||||
</div>
|
||||
<div class="md-source__repository">
|
||||
schmelczer/great-ai
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href=".." class="md-nav__link">
|
||||
Overview
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../reference/" class="md-nav__link">
|
||||
Reference
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
|
||||
<div class="md-sidebar__scrollwrap">
|
||||
<div class="md-sidebar__inner">
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Table of contents
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#install" class="md-nav__link">
|
||||
Install
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Install">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#system-dependencies" class="md-nav__link">
|
||||
System dependencies
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#install-dependencies" class="md-nav__link">
|
||||
Install dependencies
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#execute" class="md-nav__link">
|
||||
Execute
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="md-content" data-md-component="content">
|
||||
<article class="md-content__inner md-typeset">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/great_ai_example-main/README.md" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<h1 id="train-a-domain-classifier-on-the-semantic-scholar-dataset">Train a domain classifier on the <a href="https://api.semanticscholar.org/corpus">semantic scholar dataset</a><a class="headerlink" href="#train-a-domain-classifier-on-the-semantic-scholar-dataset" title="Permanent link">#</a></h1>
|
||||
<p><img alt="steps" src="diagrams/scope.svg" /></p>
|
||||
<h2 id="install">Install<a class="headerlink" href="#install" title="Permanent link">#</a></h2>
|
||||
<h3 id="system-dependencies">System dependencies<a class="headerlink" href="#system-dependencies" title="Permanent link">#</a></h3>
|
||||
<p>Make sure you have <code>python3</code>, <code>pip</code>, and <code>venv</code> installed.</p>
|
||||
<blockquote>
|
||||
<p>On Ubuntu, execute: <code>sudo apt install -y python3 python3-pip python3-venv</code></p>
|
||||
</blockquote>
|
||||
<h3 id="install-dependencies">Install dependencies<a class="headerlink" href="#install-dependencies" title="Permanent link">#</a></h3>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a>python3 -m venv --copies .env
|
||||
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="nb">source</span> .env/bin/activate
|
||||
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a>
|
||||
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a>pip install -r requirements.txt
|
||||
</code></pre></div>
|
||||
<h2 id="execute">Execute<a class="headerlink" href="#execute" title="Permanent link">#</a></h2>
|
||||
<ul>
|
||||
<li><a href="src/data/">Part 1</a></li>
|
||||
<li><a href="src/train/">Part 2</a></li>
|
||||
<li><a href="src/deploy/">Part 3</a></li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
</article>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</main>
|
||||
|
||||
<footer class="md-footer">
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
|
||||
|
||||
Made with
|
||||
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
|
||||
Material for MkDocs
|
||||
</a>
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</footer>
|
||||
|
||||
</div>
|
||||
<div class="md-dialog" data-md-component="dialog">
|
||||
<div class="md-dialog__inner md-typeset"></div>
|
||||
</div>
|
||||
<script id="__config" type="application/json">{"base": "..", "features": ["content.code.annotate", "content.tooltips"], "search": "../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
|
||||
|
||||
<script src="../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
great_ai
|
||||
notebook
|
||||
nbformat
|
||||
nbconvert
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
mongo_connection_string=mongodb://localhost:27017/ # change this
|
||||
mongo_database=great_ai_example # this will be automatically created
|
||||
|
|
@ -1,87 +0,0 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;226m2022-07-08 10:26:12,806 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-08 10:26:12,807 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-08 10:26:12,807 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-08 10:26:12,808 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-08 10:26:12,808 | INFO | Settings: configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-08 10:26:12,809 | INFO | 🔩 tracing_database: ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-08 10:26:12,809 | INFO | 🔩 large_file_implementation: LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-08 10:26:12,811 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-08 10:26:12,811 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-08 10:26:12,812 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-08 10:26:12,813 | INFO | 🔩 dashboard_table_size: 20\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-08 10:26:12,813 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-08 10:26:12,814 | WARNING | > Find out more at https://se-ml.github.io/practices/\u001b[0m\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from great_ai import GreatAI\n",
|
||||
"from asyncio import sleep\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@GreatAI.create\n",
|
||||
"async def hello_world(name: str) -> str:\n",
|
||||
" await sleep(1)\n",
|
||||
" return f\"Hello {name}!\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Trace(trace_id='27c5e581-ad6e-4d50-bb77-862dfb7963fb', created='2022-07-08T08:26:12.911975', original_execution_time_ms=1.933, logged_values={'arg:name:value': 'hi', 'arg:name:length': 2}, models=[], exception=None, output='Hello hi!', feedback=None, tags=['hello_world', 'online', 'development'])"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"await hello_world('hi')"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.10.4 ('.env': venv)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.4"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "02dd6d3afbfa9fbbe1037d64ad9014965528a1ccad21929d6e72f466389a68ad"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
from great_ai import GreatAI
|
||||
|
||||
|
||||
@GreatAI.create
|
||||
def hello_world(name: str) -> str:
|
||||
"""Learn more about GreatAI at https://github.com/schmelczer/great-ai"""
|
||||
|
||||
return f"Hello {name}!"
|
||||
|
|
@ -12,12 +12,12 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../assets/images/favicon.png">
|
||||
<link rel="icon" href="../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
<title>How-To Guides - GreatAI documentation</title>
|
||||
<title>Index - GreatAI documentation</title>
|
||||
|
||||
|
||||
|
||||
|
|
@ -98,7 +98,7 @@
|
|||
<div class="md-header__topic" data-md-component="header-topic">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
How-To Guides
|
||||
Index
|
||||
|
||||
</span>
|
||||
</div>
|
||||
|
|
@ -244,13 +244,74 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -258,19 +319,105 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active">
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
How-To Guides
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
|
@ -305,21 +452,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -351,13 +483,13 @@
|
|||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides.md" title="Edit this page" class="md-content__button md-icon">
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/index.md" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<h1>How-To Guides</h1>
|
||||
<h1>Index</h1>
|
||||
|
||||
|
||||
|
||||
|
|
@ -366,7 +498,7 @@
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -383,42 +515,6 @@
|
|||
|
||||
<footer class="md-footer">
|
||||
|
||||
|
||||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
<a href="../tutorials/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Tutorials" rel="prev">
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</div>
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Previous
|
||||
</span>
|
||||
Tutorials
|
||||
</div>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="../reference/" class="md-footer__link md-footer__link--next" aria-label="Next: Reference" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
Reference
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
</nav>
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
|
|
|
|||
|
|
@ -12,19 +12,19 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../assets/images/favicon.png">
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
<title>Hello world - GreatAI documentation</title>
|
||||
<title>Scraping - GreatAI documentation</title>
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
|
||||
|
||||
|
||||
|
|
@ -40,9 +40,9 @@
|
|||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../assets/_mkdocstrings.css">
|
||||
<link rel="stylesheet" href="../../assets/_mkdocstrings.css">
|
||||
|
||||
<script>__md_scope=new URL("..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
<script>__md_scope=new URL("../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
|
||||
|
||||
|
||||
|
|
@ -79,7 +79,7 @@
|
|||
|
||||
<header class="md-header" data-md-component="header">
|
||||
<nav class="md-header__inner md-grid" aria-label="Header">
|
||||
<a href=".." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
<a href="../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
|
@ -98,7 +98,7 @@
|
|||
<div class="md-header__topic" data-md-component="header-topic">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Hello world
|
||||
Scraping
|
||||
|
||||
</span>
|
||||
</div>
|
||||
|
|
@ -199,7 +199,7 @@
|
|||
|
||||
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
|
||||
<label class="md-nav__title" for="__drawer">
|
||||
<a href=".." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
<a href="../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
|
@ -230,7 +230,7 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href=".." class="md-nav__link">
|
||||
<a href="../.." class="md-nav__link">
|
||||
Overview
|
||||
</a>
|
||||
</li>
|
||||
|
|
@ -244,13 +244,183 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -260,22 +430,7 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../reference/" class="md-nav__link">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
</a>
|
||||
</li>
|
||||
|
|
@ -290,28 +445,13 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../explanation/" class="md-nav__link">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -340,20 +480,20 @@
|
|||
<article class="md-content__inner md-typeset">
|
||||
|
||||
|
||||
<a href="hello_world.py" title="Download Notebook" class="md-content__button md-icon">
|
||||
<a href="scraping.ipynb" title="Download Notebook" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M5 20h14v-2H5m14-9h-4V3H9v6H5l7 7 7-7Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/hello_world.py" title="Edit this page" class="md-content__button md-icon">
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/scraping.ipynb" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<h1>Hello world</h1>
|
||||
<h1>Scraping</h1>
|
||||
|
||||
<script>
|
||||
(function (global, factory) {
|
||||
|
|
@ -1405,49 +1545,25 @@ Licensed under the Apache License, Version 2.0.
|
|||
</div>
|
||||
</clipboard-copy>
|
||||
</div>
|
||||
<div class="highlight-ipynb hl-python"><pre><span></span><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span>
|
||||
</pre></div>
|
||||
<div id="cell-1" class="clipboard-copy-txt">from great_ai import GreatAI</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<div class="highlight-ipynb hl-python"><pre><span></span><span class="kn">import</span> <span class="nn">httpx</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div><div class="jp-Cell jp-CodeCell jp-Notebook-cell jp-mod-noOutputs ">
|
||||
<div class="jp-Cell jp-CodeCell jp-Notebook-cell jp-mod-noOutputs ">
|
||||
<div class="jp-Cell-inputWrapper">
|
||||
<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
|
||||
</div>
|
||||
<div class="jp-InputArea jp-Cell-inputArea">
|
||||
<div class="jp-InputPrompt jp-InputArea-prompt">In [ ]:</div><div class="jp-CodeMirrorEditor jp-Editor jp-InputArea-editor" data-type="inline">
|
||||
|
||||
|
||||
<div class="CodeMirror cm-s-jupyter">
|
||||
<div class="zeroclipboard-container">
|
||||
<clipboard-copy for="cell-2">
|
||||
<div>
|
||||
<span class="notice" hidden>Copied!</span>
|
||||
<svg aria-hidden="true" width="20" height="20" viewBox="0 0 16 16" version="1.1" data-view-component="true" class="clipboard-copy-icon">
|
||||
<path fill="currentColor" fill-rule="evenodd" d="M0 6.75C0 5.784.784 5 1.75 5h1.5a.75.75 0 010 1.5h-1.5a.25.25 0 00-.25.25v7.5c0 .138.112.25.25.25h7.5a.25.25 0 00.25-.25v-1.5a.75.75 0 011.5 0v1.5A1.75 1.75 0 019.25 16h-7.5A1.75 1.75 0 010 14.25v-7.5z"></path>
|
||||
<path fill="currentColor" fill-rule="evenodd" d="M5 1.75C5 .784 5.784 0 6.75 0h7.5C15.216 0 16 .784 16 1.75v7.5A1.75 1.75 0 0114.25 11h-7.5A1.75 1.75 0 015 9.25v-7.5zm1.75-.25a.25.25 0 00-.25.25v7.5c0 .138.112.25.25.25h7.5a.25.25 0 00.25-.25v-7.5a.25.25 0 00-.25-.25h-7.5z"></path>
|
||||
</svg>
|
||||
</div>
|
||||
</clipboard-copy>
|
||||
</div>
|
||||
<div class="highlight-ipynb hl-python"><pre><span></span><span class="nd">@GreatAI</span><span class="o">.</span><span class="n">create</span>
|
||||
<span class="k">def</span> <span class="nf">hello_world</span><span class="p">(</span><span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
<span class="sd">"""Learn more about GreatAI at https://github.com/schmelczer/great-ai"""</span>
|
||||
|
||||
<span class="k">return</span> <span class="sa">f</span><span class="s2">"Hello </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">!"</span>
|
||||
<span class="n">transport</span> <span class="o">=</span> <span class="n">httpx</span><span class="o">.</span><span class="n">AsyncHTTPTransport</span><span class="p">(</span><span class="n">retries</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">limits</span> <span class="o">=</span> <span class="n">httpx</span><span class="o">.</span><span class="n">Limits</span><span class="p">(</span><span class="n">max_connections</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
|
||||
<span class="n">timeout</span> <span class="o">=</span> <span class="n">httpx</span><span class="o">.</span><span class="n">Timeout</span><span class="p">(</span><span class="n">connect</span><span class="o">=</span><span class="mf">60.0</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="mi">300</span><span class="p">,</span> <span class="n">write</span><span class="o">=</span><span class="mi">60</span><span class="p">,</span> <span class="n">pool</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
|
||||
<span class="k">async</span> <span class="k">with</span> <span class="n">httpx</span><span class="o">.</span><span class="n">AsyncClient</span><span class="p">(</span>
|
||||
<span class="n">transport</span><span class="o">=</span><span class="n">transport</span><span class="p">,</span> <span class="n">limits</span><span class="o">=</span><span class="n">limits</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="n">timeout</span>
|
||||
<span class="p">)</span> <span class="k">as</span> <span class="n">client</span><span class="p">:</span>
|
||||
<span class="k">pass</span>
|
||||
</pre></div>
|
||||
<div id="cell-2" class="clipboard-copy-txt">@GreatAI.create
|
||||
def hello_world(name: str) -> str:
|
||||
"""Learn more about GreatAI at https://github.com/schmelczer/great-ai"""
|
||||
|
||||
return f"Hello {name}!"</div>
|
||||
<div id="cell-1" class="clipboard-copy-txt">import httpx
|
||||
|
||||
transport = httpx.AsyncHTTPTransport(retries=2)
|
||||
limits = httpx.Limits(max_connections=None)
|
||||
timeout = httpx.Timeout(connect=60.0, read=300, write=60, pool=None)
|
||||
async with httpx.AsyncClient(
|
||||
transport=transport, limits=limits, timeout=timeout
|
||||
) as client:
|
||||
pass</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -1466,7 +1582,7 @@ def hello_world(name: str) -> str:
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -1503,10 +1619,10 @@ def hello_world(name: str) -> str:
|
|||
<div class="md-dialog" data-md-component="dialog">
|
||||
<div class="md-dialog__inner md-typeset"></div>
|
||||
</div>
|
||||
<script id="__config" type="application/json">{"base": "..", "features": ["content.code.annotate", "content.tooltips"], "search": "../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
<script id="__config" type="application/json">{"base": "../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
|
||||
|
||||
<script src="../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
<script src="../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
29
how-to-guides/scraping/scraping.ipynb
Normal file
29
how-to-guides/scraping/scraping.ipynb
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import httpx\n",
|
||||
"\n",
|
||||
"transport = httpx.AsyncHTTPTransport(retries=2)\n",
|
||||
"limits = httpx.Limits(max_connections=None)\n",
|
||||
"timeout = httpx.Timeout(connect=60.0, read=300, write=60, pool=None)\n",
|
||||
"async with httpx.AsyncClient(\n",
|
||||
" transport=transport, limits=limits, timeout=timeout\n",
|
||||
") as client:\n",
|
||||
" pass"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
230
ideas.drawio
230
ideas.drawio
File diff suppressed because one or more lines are too long
246
index.html
246
index.html
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="assets/images/favicon.png">
|
||||
<link rel="icon" href="media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -70,7 +70,7 @@
|
|||
<div data-md-component="skip">
|
||||
|
||||
|
||||
<a href="#overview-of-greatai" class="md-skip">
|
||||
<a href="#features" class="md-skip">
|
||||
Skip to content
|
||||
</a>
|
||||
|
||||
|
|
@ -241,8 +241,6 @@
|
|||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link md-nav__link--active" for="__toc">
|
||||
Overview
|
||||
|
|
@ -259,8 +257,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
|
|
@ -294,6 +290,13 @@
|
|||
Why GreatAI?
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#production-use" class="md-nav__link">
|
||||
Production use
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
|
@ -311,13 +314,74 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -326,13 +390,107 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
<a href="simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -364,21 +522,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -396,8 +539,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
|
|
@ -431,6 +572,13 @@
|
|||
Why GreatAI?
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#production-use" class="md-nav__link">
|
||||
Production use
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
|
@ -454,7 +602,11 @@
|
|||
|
||||
|
||||
|
||||
<h1 id="overview-of-greatai">Overview of GreatAI<a class="headerlink" href="#overview-of-greatai" title="Permanent link">#</a></h1>
|
||||
<div style="display: flex; justify-content: space-between; align-items: center;">
|
||||
<h1 style="margin: 0">Overview of GreatAI</h1>
|
||||
<img src="media/logo.png" width=80>
|
||||
</div>
|
||||
|
||||
<p><a href="https://github.com/schmelczer/great-ai/actions/workflows/check.yml"><img alt="Test" src="https://github.com/schmelczer/great-ai/actions/workflows/test.yml/badge.svg" /></a>
|
||||
<a href="https://sonar.schmelczer.com/dashboard?id=great-ai"><img alt="Quality Gate Status" src="https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=alert_status" /></a>
|
||||
<a href="https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml"><img alt="Publish on PyPI" src="https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml/badge.svg" /></a>
|
||||
|
|
@ -464,15 +616,15 @@
|
|||
<details class="quote">
|
||||
<summary>Case studies</summary>
|
||||
<p>"There is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature."
|
||||
— <a href="https://ieeexplore.ieee.org/abstract/document/9359253">John et al.</a></p>
|
||||
<p>"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." — <a href="https://link.springer.com/article/10.1007/s10664-021-09993-1">Haakman et al.</a></p>
|
||||
<p>"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." — <a href="https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf">Sculley et al.</a></p>
|
||||
<p>"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. — <a href="https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw">Serban et al.</a></p>
|
||||
— <a href="https://ieeexplore.ieee.org/abstract/document/9359253" target="_blank">John et al.</a></p>
|
||||
<p>"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." — <a href="https://link.springer.com/article/10.1007/s10664-021-09993-1" target="_blank">Haakman et al.</a></p>
|
||||
<p>"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." — <a href="https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf" target="_blank">Sculley et al.</a></p>
|
||||
<p>"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. — <a href="https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw" target="_blank">Serban et al.</a></p>
|
||||
</details>
|
||||
<h2 id="features">Features<a class="headerlink" href="#features" title="Permanent link">#</a></h2>
|
||||
<ul class="task-list">
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save prediction traces of each prediction including arguments and model versions</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save feedback and merge it into a ground-truth database</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save feedback and merge it into a ground-truth database <em><img alt="➡" class="twemoji" src="https://twemoji.maxcdn.com/v/latest/svg/27a1.svg" title=":arrow_right:" /> quasi-shadow deployment</em></li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Version and store models and data on shared infrastructure <em>(MongoDB GridFS, S3-compatible storage, shared local-volume)</em></li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Input validation</li>
|
||||
|
|
@ -482,11 +634,11 @@
|
|||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Well-tested utilities for common NLP tasks (cleaning, language-tagging, sentence-segmentation, etc.)</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> A simple, unified configuration interface</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Fully-typed API for IntelliSense support</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Fully-typed API for <a href="https://github.com/microsoft/pylance-release" target="_blank">Pylance</a> and <a href="http://mypy-lang.org" target="_blank">MyPy</a> support</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Auto-reload for development</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Deployable Jupyter Notebooks</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Docker support for deployment</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Dashboard for high-level overview and searching traces</li>
|
||||
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled/><span class="task-list-indicator"></span></label> Shadow deployment</li>
|
||||
</ul>
|
||||
<h2 id="hello-world">Hello world<a class="headerlink" href="#hello-world" title="Permanent link">#</a></h2>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a>pip install great-ai
|
||||
|
|
@ -517,7 +669,7 @@
|
|||
<div class="highlight"><span class="filename">terminal</span><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a>great-ai hello-world.py
|
||||
</code></pre></div>
|
||||
<blockquote>
|
||||
<p>Navigate to <a href="http://127.0.0.1:6060/">localhost:6060</a> in your browser.</p>
|
||||
<p>Navigate to <a href="http://127.0.0.1:6060">localhost:6060</a> in your browser.</p>
|
||||
</blockquote>
|
||||
<div style="display: flex; justify-content: space-evenly;">
|
||||
<p><img alt="" loading="lazy" src="media/hello-world-dashboard.png" /></p>
|
||||
|
|
@ -525,11 +677,11 @@
|
|||
</div>
|
||||
<div class="admonition success">
|
||||
<p class="admonition-title">Success</p>
|
||||
<p>Your GreatAI service is ready for production use. Many of the <a href="https://se-ml.github.io/" target="_blank">SE4ML best-practices</a> are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.</p>
|
||||
<p>Your GreatAI service is ready for production use. Many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best-practices</a> are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.</p>
|
||||
</div>
|
||||
<h2 id="why-is-this-great">Why is this GREAT?<a class="headerlink" href="#why-is-this-great" title="Permanent link">#</a></h2>
|
||||
<p><img alt="scope of GreatAI" src="scope-simple.drawio.svg" /></p>
|
||||
<p>GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented concerning the following 5 aspects:</p>
|
||||
<p><img alt="scope of GreatAI" src="media/scope-simple.drawio.svg" /></p>
|
||||
<p>GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented aiming to achieve the following attributes:</p>
|
||||
<ul>
|
||||
<li><strong>G</strong>eneral: use any Python library without restriction</li>
|
||||
<li><strong>R</strong>obust: have error-handling and well-tested utilities out-of-the-box </li>
|
||||
|
|
@ -538,19 +690,23 @@
|
|||
<li><strong>T</strong>rustworthy: deploy models that you and society can confidently trust</li>
|
||||
</ul>
|
||||
<h2 id="why-greatai">Why GreatAI?<a class="headerlink" href="#why-greatai" title="Permanent link">#</a></h2>
|
||||
<p>There are other, existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker/" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.</p>
|
||||
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many <a href="https://se-ml.github.io/" target="_blank">best-practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
|
||||
<p>There are other, existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.</p>
|
||||
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many <a href="https://se-ml.github.io" target="_blank">best-practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
|
||||
<div style="display: flex; justify-content: space-evenly;">
|
||||
<p><a class="md-button md-button--primary" href="https://pypi.org/project/great-ai/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg></span> Find it on PyPI</a></p>
|
||||
<p><a class="md-button md-button--primary" href="https://pypi.org/project/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg></span> Find it on PyPI</a></p>
|
||||
<p><a class="md-button md-button--primary" href="https://hub.docker.com/repository/docker/schmelczera/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M349.9 236.3h-66.1v-59.4h66.1v59.4zm0-204.3h-66.1v60.7h66.1V32zm78.2 144.8H362v59.4h66.1v-59.4zm-156.3-72.1h-66.1v60.1h66.1v-60.1zm78.1 0h-66.1v60.1h66.1v-60.1zm276.8 100c-14.4-9.7-47.6-13.2-73.1-8.4-3.3-24-16.7-44.9-41.1-63.7l-14-9.3-9.3 14c-18.4 27.8-23.4 73.6-3.7 103.8-8.7 4.7-25.8 11.1-48.4 10.7H2.4c-8.7 50.8 5.8 116.8 44 162.1 37.1 43.9 92.7 66.2 165.4 66.2 157.4 0 273.9-72.5 328.4-204.2 21.4.4 67.6.1 91.3-45.2 1.5-2.5 6.6-13.2 8.5-17.1l-13.3-8.9zm-511.1-27.9h-66v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm-78.1-72.1h-66.1v60.1h66.1v-60.1z"/></svg></span> Find it on DockerHub</a></p>
|
||||
</div>
|
||||
<h2 id="production-use">Production use<a class="headerlink" href="#production-use" title="Permanent link">#</a></h2>
|
||||
<p>GreatAI has been battle-tested on the core platform services of <a href="https://www.scoutinscience.com/" target="_blank">ScoutinScience</a>.</p>
|
||||
<p><img alt="ScoutinScience logo" src="media/scoutinscience.svg#only-light" />
|
||||
<img alt="ScoutinScience logo" src="media/scoutinscience-white.svg#only-dark" /></p>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -572,13 +728,13 @@
|
|||
|
||||
|
||||
|
||||
<a href="tutorials/" class="md-footer__link md-footer__link--next" aria-label="Next: Tutorials" rel="next">
|
||||
<a href="tutorial/" class="md-footer__link md-footer__link--next" aria-label="Next: Train and deploy a SOTA model" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
Tutorials
|
||||
Train and deploy a SOTA model
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
|
|
|
|||
45
index.md
45
index.md
|
|
@ -1,29 +1,30 @@
|
|||
# Overview of GreatAI
|
||||
<div style="display: flex; justify-content: space-between; align-items: center;">
|
||||
<h1 style="margin: 0">Overview of GreatAI</h1>
|
||||
<img src="media/logo.png" width=80>
|
||||
</div>
|
||||
|
||||
[](https://github.com/schmelczer/great-ai/actions/workflows/check.yml)
|
||||
[](https://sonar.schmelczer.com/dashboard?id=great-ai)
|
||||
[](https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml)
|
||||
[](https://github.com/schmelczer/great-ai/actions/workflows/docker.yaml)
|
||||
[](https://pepy.tech/project/great-ai)
|
||||
|
||||
|
||||
Applying AI is becoming increasingly easier but many case studies have shown that these applications are often deployed poorly. This may lead to suboptimal performance and to introducing [unintended biases](https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction){ target=_blank }. To extend the list of available solutions, ==GreatAI helps you easily transform your prototype AI code into production-ready software.==
|
||||
|
||||
??? quote "Case studies"
|
||||
"There is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature."
|
||||
— [John et al.](https://ieeexplore.ieee.org/abstract/document/9359253)
|
||||
— [John et al.](https://ieeexplore.ieee.org/abstract/document/9359253){ target=_blank }
|
||||
|
||||
"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." — [Haakman et al.](https://link.springer.com/article/10.1007/s10664-021-09993-1)
|
||||
|
||||
"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." — [Sculley et al.](https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf)
|
||||
|
||||
"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. — [Serban et al.](https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw)
|
||||
"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." — [Haakman et al.](https://link.springer.com/article/10.1007/s10664-021-09993-1){ target=_blank }
|
||||
|
||||
"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." — [Sculley et al.](https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf){ target=_blank }
|
||||
|
||||
"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. — [Serban et al.](https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw){ target=_blank }
|
||||
|
||||
## Features
|
||||
|
||||
- [x] Save prediction traces of each prediction including arguments and model versions
|
||||
- [x] Save feedback and merge it into a ground-truth database
|
||||
- [x] Save feedback and merge it into a ground-truth database *:arrow_right: quasi-shadow deployment*
|
||||
- [x] Version and store models and data on shared infrastructure *(MongoDB GridFS, S3-compatible storage, shared local-volume)*
|
||||
- [x] Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration
|
||||
- [x] Input validation
|
||||
|
|
@ -33,11 +34,11 @@ Applying AI is becoming increasingly easier but many case studies have shown tha
|
|||
- [x] Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing
|
||||
- [x] Well-tested utilities for common NLP tasks (cleaning, language-tagging, sentence-segmentation, etc.)
|
||||
- [x] A simple, unified configuration interface
|
||||
- [x] Fully-typed API for IntelliSense support
|
||||
- [x] Fully-typed API for [Pylance](https://github.com/microsoft/pylance-release){ target=_blank } and [MyPy](http://mypy-lang.org){ target=_blank } support
|
||||
- [x] Auto-reload for development
|
||||
- [x] Deployable Jupyter Notebooks
|
||||
- [x] Docker support for deployment
|
||||
- [x] Dashboard for high-level overview and searching traces
|
||||
- [ ] Shadow deployment
|
||||
|
||||
## Hello world
|
||||
|
||||
|
|
@ -66,7 +67,7 @@ def hello_world(name: str) -> str: #(2)
|
|||
```sh title="terminal"
|
||||
great-ai hello-world.py
|
||||
```
|
||||
> Navigate to [localhost:6060](http://127.0.0.1:6060/) in your browser.
|
||||
> Navigate to [localhost:6060](http://127.0.0.1:6060) in your browser.
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||
{ loading=lazy }
|
||||
|
|
@ -75,14 +76,13 @@ great-ai hello-world.py
|
|||
</div>
|
||||
|
||||
!!! success
|
||||
Your GreatAI service is ready for production use. Many of the [SE4ML best-practices](https://se-ml.github.io/){ target=_blank } are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.
|
||||
|
||||
Your GreatAI service is ready for production use. Many of the [SE4ML best-practices](https://se-ml.github.io){ target=_blank } are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.
|
||||
|
||||
## Why is this GREAT?
|
||||
|
||||
|
||||
|
||||
|
||||
GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented concerning the following 5 aspects:
|
||||
GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented aiming to achieve the following attributes:
|
||||
|
||||
- **G**eneral: use any Python library without restriction
|
||||
- **R**obust: have error-handling and well-tested utilities out-of-the-box
|
||||
|
|
@ -92,13 +92,20 @@ GreatAI fits between the prototype and deployment phase of your (or your organis
|
|||
|
||||
## Why GreatAI?
|
||||
|
||||
There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker/){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.
|
||||
There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.
|
||||
|
||||
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many [best-practices](https://se-ml.github.io/){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
|
||||
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many [best-practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
|
||||
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||
[:fontawesome-brands-python: Find it on PyPI](https://pypi.org/project/great-ai/){ .md-button .md-button--primary }
|
||||
[:fontawesome-brands-python: Find it on PyPI](https://pypi.org/project/great-ai){ .md-button .md-button--primary }
|
||||
|
||||
[:fontawesome-brands-docker: Find it on DockerHub](https://hub.docker.com/repository/docker/schmelczera/great-ai){ .md-button .md-button--primary }
|
||||
</div>
|
||||
|
||||
## Production use
|
||||
|
||||
GreatAI has been battle-tested on the core platform services of [ScoutinScience](https://www.scoutinscience.com/){ target=_blank }.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
BIN
media/favicon.ico
Normal file
BIN
media/favicon.ico
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
BIN
media/logo.png
Normal file
BIN
media/logo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 84 KiB |
|
|
@ -1,4 +1,4 @@
|
|||
<svg host="65bd71144e" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="1181px" height="281px" viewBox="-0.5 -0.5 1181 281" content="<mxfile><diagram id="ccudx8biEBM-x_P_p45R" name="Page-1">1VrbcuI4EP0aqpIHUr5gA49sYGa3KqlJFbs1k0fZFkYV2/JIIsH79duy5bsJBhySzUMstaS2pD59uiUzMu/D/XeG4u0j9XAwMjRvPzKXI8MwdMuAh5QkmcSaK4HPiJeJ9FKwJv9iJdSUdEc8zGsdBaWBIHFd6NIowq6oyRBj9K3ebUOD+ltj5OOWYO2ioC39STyxzaQzY1rK/8TE3+Zv1u151hKivLNaCd8ij75VROZqZN4zSkVWCvf3OJCbl+9LNu7bgdZiYgxHos8ANWMuknxt2IOlqmpEI3j8sRVhADUdihsaCWUPfQL19vvUFDjdMVepVMYViPlY9ZpkIvmyyjA1x++YhliwBDowHCBBXuv7jpT5/KJfuUIoqEV2L1jN5RUFO6V0ZNiBUEurbYX9e0fzhjFPF72ADvok3peNUPLl84nhmFEXc449iWwkUK4YppTpznq2dhwwEMuimwQk8jAzodPblgi8jlG6hW/gRXUzOHQHPb0HpxAg98VnUvpjJ0ANVnKubGW9Z6tXzATev2sH1WrO7WyIclwzx/Fb6QY6vCyVbasuoGmX284+EaywTpb8goqWV55l5c7Kq8t9tXGZVGtPmBGYIWZKeBbyzTbyjc9CvnkQ+U4JzSFcYZliX0Pu7x3hRBAaHfSEQtyagsNyyY3St4p8wDVYJfJvD/oQ2AMFAQ6oz1AIg+OKFWttFfMec7YN2eM8COnD+JE+0+p+ZHb4Ud6n6kezAdxodg3On7SRb30W8ifXQv5qHweUoQzyYCkbhRJL2f82sosOkcPlI1SJkrMjgQcwH8Rt1i7BkYs/12U2JAjuKexN+mpTd5CODbWrFfkm/ZNhSzD6gistmmavFt+GcT7LagSxLuezP8j5dPNKQazptR7iW+zlSvdE/CrVQO250lJqlJWkUmkGxV5UYLWpYD40FaRDF4yhpNIhpiQSvKL5SQpKIMyMOgtPZlrDlJnG0rDF1HrZ2vqgXPNvhiASyjQzZQze5JEu7tEgOxVUJODyMMFog5lkhfSM5OGvlql+KbqYTqbHc97JR+W8+uTr8oV2Nb7ID/4DE0aLEaZ2gxGMhgmzeapR1XN1Q5Gl1xVZzdiRra6l6FIKM+xhKUy3j+dOSwxZT5I6BPiDhCqKPMS84wkPZT6KCE8TJiRfCzPbUo/3y3BOJlOji0ybc7pZJ3zhhQQyONgd7UfMs8ISv0L59lSqvEp21TdBP+GAok+OJ0kFOQ2dJBWscO4RpZO88nKFDw+T17lXB+eSnj7vYD37Oqw3s+okMm0Grt6s1zjW2no/1juDmDov8hqYkWlG6tn0ATk4x0p/7ygukJGTq9S6DaBGzZts3Ok0HSeLQbxmfpyq/+Ey5UPydLr4C/55KXWHODVim2HXCK3hrU/pQ7smK988PqweM+JdvLwQmhWxcO9u29nu/5GRC/wep+QcTB15ZyeY7CHSTu09MF1wcIn4hrIwvdyAY4igx5MEOLh4O1emCGOGkSd3hGP2SlzcD5AApSxwfxE8GJfi4Sr2v/SS8PLrhiICl0H3udp2UQQuosWREPwxFxWtwDmeWt3Z1qkReKw3FPU8d5wTgbs+KJyMmfzQ7uEN2gUlcTd2/ALrXeGCWT8cei+76VkwQTbIFWkqEjokyohTbGUYz6+L1bFLxvKEqljuYC7GMYOhwJP81BB5/S+RJ9Bf/iuC6V0d6vOOaxnro65liu/3p1/LnEhnfb5FnsOvPb1Ka3uVPjgpHth1qJa/hMgYqvw9ibn6Dw==</diagram></mxfile>">
|
||||
<svg host="65bd71144e" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="1181px" height="281px" viewBox="-0.5 -0.5 1181 281" content="<mxfile><diagram id="ccudx8biEBM-x_P_p45R" name="Page-1">3VrbcuI4EP0aqpKHpHzBBh7ZwMxuVVKTKnZrJo+yJUAV2/JIIsH79duy5bsJ5jqZ5QFLLaktqfucbgkG9kO4/cpRvH5imAQDy8DbgT0bWJZpWSN4KEmSScYjLVhxinWnUrCg/xItNLR0QzERtY6SsUDSuC70WRQRX9ZkiHP2Xu+2ZEH9rTFakZZg4aOgLf1OsVzrVeTLUvI/CV2t8zeb7iRrCVHeWa9ErBFm7xWRPR/YD5wxmZXC7QMJ1Obl+5KN+7KjtZgYJ5HsM0DPWMgkXxvBsFRdjVgEjz/WMgygZkJxySKp7WEOod5+n56CYBvua5WWNhHiK6J7DTORelllmJ7jV8JCInkCHTgJkKRv9X1H2nyrol+5QijoRXYvWM/lDQUbrXRguYHUS6tthftzw/KGO5EuegodzGG8LRuhtFLPZ05iznwiBMHKs5FEuWKYUqY769nacfCBWBX9JKARJtyGTu9rKskiRukWvgOK6mbw2AZ64kevECD/dcWV9NtGghqi5ULbyvnIVm+ES7L90A661Z642RANXDv34/cSBia8LJWtqxAwjNNt5x7orLBOnvyAipFXXlTl3smrs221cZZUa8+EU5gh4Vp4lOfbbc+3fpXn2zs93ytd8xxQmKW+byD/54YKKimLdiKhELem4PFccqP1zaMV+DVYJVrd7sQQ2AMFAQnYiqMQBscVK9baKubdB7Yl3ZI8CJnnwZE5Nuo4sjtwlPep4mh8BhiNr8H5w7bnO7/K84fX8vz5Ng4YR5nLg6VcFCpfyr7bnl10iDyhHqFOlLwNDTC4+Vlgs/ApiXzy20PGcRqhpwsy7oUgY9pXCj1NrGEk1gTnSrdU/ijVQO2l0lJqVJWkUmmGsl4AdtoAnpwbwOnQKecoqXSIGY2kqGh+VoLSEcZWnTuHY6Nhykxjadhiar1s7VwoQ/ybI4hfKjlMcS6a6O9iDANySslkAkCFCUZLwhWW05MNJv/D/HI0HO3PL4eXyi/N4edFuXE1lOeH7DPDvIXjkdvAsdUwYTZPPap6hm0ocsy6IqfJ+NnqWopOJR7LPS/xmO7+PGVGIMNIUkAAHpSroggjjvcnF4yvUERFmpwg9VqY2Zph0S+bOJgCrS4KbM7pZpGIKQ4pZEuwO8a3WGSFGXmD8u2hBHeNTKZ3MnzAYcAc7k9tCnI6d2pTsMKxx4FO8srLFT7cTV7HHtOPJT1z0sF67nVYb+zUSWTUDFy9Wa9xhHTNfqx3BDF1Xpo1fEYlBymy2SPySO4r/dFRXNYiL1dpdBtAj5o02bgTNB3ngbOgZrKfqv8RKlFD6iQ4/Qu+cErdIUmN2GbYBUILeOtz+jCuyco3T4/zp4x4p6+vlGVFIv3723aO+jsycuG/+yk5d6aOvLPTmdxzpJ3GR850wnEjEkvGw/QiAQ4Pku1PEuC4gTe+ShHuOEFY7Ygg/I36pJ9DgitlgfuT+IOVtgfBAwsYT6dimx4yiaX7VeTL9KOONJKzV1JpMQx3Pv1yqmddxZNOvdo7/bqhiOVl+H6ptp0Uy4u4syeYX+aiohWC70ZOd952aCy/MxuKep5gjonlXT8DHOwzOTowWaJNID+XsXqjZXcQP+2mZ8olXSJfpklN6NEoo2C5VglBfsmrD3AqK0iYzgo8IuRdzGEoMK44NNhe+n7nkxCpbrVG93XQTDquipxLXRUVv98fflV0IDH2+S3yGKbuCVijDVjzWoiFavlPiIzryv+T2PP/AA==</diagram></mxfile>">
|
||||
<defs/>
|
||||
<g>
|
||||
<path d="M 331 140 L 379.63 140" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/>
|
||||
|
|
@ -18,7 +18,7 @@
|
|||
</div>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<text x="274" y="156" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
<text x="274" y="157" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
Preprocessed data
|
||||
</text>
|
||||
</switch>
|
||||
|
|
@ -50,13 +50,13 @@
|
|||
</g>
|
||||
<path d="M 526 140 L 560.63 140" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/>
|
||||
<path d="M 565.88 140 L 558.88 143.5 L 560.63 140 L 558.88 136.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
|
||||
<path d="M 376 180 L 396 100 L 536 100 L 516 180 Z" fill="#1ba1e2" stroke="#006eaf" stroke-miterlimit="10" pointer-events="all"/>
|
||||
<path d="M 376 180 L 396 100 L 536 100 L 516 180 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
|
||||
<g transform="translate(-0.5 -0.5)">
|
||||
<switch>
|
||||
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
|
||||
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 158px; height: 1px; padding-top: 140px; margin-left: 377px;">
|
||||
<div data-drawio-colors="color: #ffffff; " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(255, 255, 255); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
|
||||
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
|
||||
<b>
|
||||
<font style="font-size: 14px">
|
||||
Exploration &
|
||||
|
|
@ -70,21 +70,21 @@
|
|||
</div>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<text x="456" y="144" fill="#ffffff" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
<text x="456" y="144" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
Exploration &...
|
||||
</text>
|
||||
</switch>
|
||||
</g>
|
||||
<path d="M 639.5 190 L 639.92 240 Q 640 250 650 250 L 743.63 250" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" stroke-dasharray="3 3" pointer-events="stroke"/>
|
||||
<path d="M 748.88 250 L 741.88 253.5 L 743.63 250 L 741.88 246.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
|
||||
<path d="M 567 105 C 567 96.72 599.46 90 639.5 90 C 658.73 90 677.17 91.58 690.77 94.39 C 704.36 97.21 712 101.02 712 105 L 712 175 C 712 183.28 679.54 190 639.5 190 C 599.46 190 567 183.28 567 175 Z" fill="#1ba1e2" stroke="#006eaf" stroke-miterlimit="10" pointer-events="all"/>
|
||||
<path d="M 712 105 C 712 113.28 679.54 120 639.5 120 C 599.46 120 567 113.28 567 105" fill="none" stroke="#006eaf" stroke-miterlimit="10" pointer-events="all"/>
|
||||
<path d="M 567 105 C 567 96.72 599.46 90 639.5 90 C 658.73 90 677.17 91.58 690.77 94.39 C 704.36 97.21 712 101.02 712 105 L 712 175 C 712 183.28 679.54 190 639.5 190 C 599.46 190 567 183.28 567 175 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
|
||||
<path d="M 712 105 C 712 113.28 679.54 120 639.5 120 C 599.46 120 567 113.28 567 105" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
|
||||
<g transform="translate(-0.5 -0.5)">
|
||||
<switch>
|
||||
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
|
||||
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 153px; margin-left: 568px;">
|
||||
<div data-drawio-colors="color: #ffffff; " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(255, 255, 255); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
|
||||
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
|
||||
<font style="font-size: 14px">
|
||||
Trained models & prototype inference code
|
||||
</font>
|
||||
|
|
@ -92,7 +92,7 @@
|
|||
</div>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<text x="640" y="156" fill="#ffffff" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
<text x="640" y="157" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
Trained models & protot...
|
||||
</text>
|
||||
</switch>
|
||||
|
|
@ -151,13 +151,13 @@
|
|||
</text>
|
||||
</switch>
|
||||
</g>
|
||||
<path d="M 740 60 L 760 0 L 950 0 L 930 60 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<path d="M 740 60 L 760 0 L 950 0 L 930 60 Z" fill="#1ba1e2" stroke="#006eaf" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<g transform="translate(-0.5 -0.5)">
|
||||
<switch>
|
||||
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
|
||||
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 208px; height: 1px; padding-top: 30px; margin-left: 741px;">
|
||||
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: none; white-space: normal; overflow-wrap: normal;">
|
||||
<div data-drawio-colors="color: #ffffff; " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(255, 255, 255); line-height: 1.2; pointer-events: none; white-space: normal; overflow-wrap: normal;">
|
||||
<b style="font-size: 14px">
|
||||
Transforming into
|
||||
<br/>
|
||||
|
|
@ -169,7 +169,7 @@
|
|||
</div>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<text x="845" y="34" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
<text x="845" y="34" fill="#ffffff" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
Transforming into...
|
||||
</text>
|
||||
</switch>
|
||||
|
|
@ -178,14 +178,14 @@
|
|||
<path d="M 845 218.88 L 841.5 211.88 L 845 213.63 L 848.5 211.88 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<path d="M 845 190 L 845 213.63" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<path d="M 845 218.88 L 841.5 211.88 L 845 213.63 L 848.5 211.88 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<path d="M 767.5 105 C 767.5 96.72 802.2 90 845 90 C 865.55 90 885.27 91.58 899.8 94.39 C 914.33 97.21 922.5 101.02 922.5 105 L 922.5 175 C 922.5 183.28 887.8 190 845 190 C 802.2 190 767.5 183.28 767.5 175 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<path d="M 922.5 105 C 922.5 113.28 887.8 120 845 120 C 802.2 120 767.5 113.28 767.5 105" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<path d="M 767.5 105 C 767.5 96.72 802.2 90 845 90 C 865.55 90 885.27 91.58 899.8 94.39 C 914.33 97.21 922.5 101.02 922.5 105 L 922.5 175 C 922.5 183.28 887.8 190 845 190 C 802.2 190 767.5 183.28 767.5 175 Z" fill="#1ba1e2" stroke="#006eaf" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<path d="M 922.5 105 C 922.5 113.28 887.8 120 845 120 C 802.2 120 767.5 113.28 767.5 105" fill="none" stroke="#006eaf" stroke-miterlimit="10" pointer-events="none"/>
|
||||
<g transform="translate(-0.5 -0.5)">
|
||||
<switch>
|
||||
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
|
||||
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 153px; height: 1px; padding-top: 153px; margin-left: 769px;">
|
||||
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: none; white-space: normal; overflow-wrap: normal;">
|
||||
<div data-drawio-colors="color: #ffffff; " style="box-sizing: border-box; font-size: 0px; text-align: center;">
|
||||
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(255, 255, 255); line-height: 1.2; pointer-events: none; white-space: normal; overflow-wrap: normal;">
|
||||
<font style="font-size: 14px">
|
||||
Artifact combining the model with deplyoment best-practices
|
||||
</font>
|
||||
|
|
@ -193,7 +193,7 @@
|
|||
</div>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<text x="845" y="156" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
<text x="845" y="157" fill="#ffffff" font-family="Helvetica" font-size="12px" text-anchor="middle">
|
||||
Artifact combining the mod...
|
||||
</text>
|
||||
</switch>
|
||||
|
Before Width: | Height: | Size: 19 KiB After Width: | Height: | Size: 19 KiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 9.8 KiB After Width: | Height: | Size: 9.8 KiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
122
media/scoutinscience-white.svg
Normal file
122
media/scoutinscience-white.svg
Normal file
|
|
@ -0,0 +1,122 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 26.2.1, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
|
||||
<svg version="1.1" id="Group_805" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 369.5 67.2" style="enable-background:new 0 0 369.5 67.2;" xml:space="preserve" fill="white">
|
||||
<style type="text/css">
|
||||
|
||||
.st0{clip-path:url(#SVGID_00000084499246685778113730000005394386986623376315_);fill:#14C9C0;}
|
||||
.st1{clip-path:url(#SVGID_00000084499246685778113730000005394386986623376315_);fill:#ffffff;}
|
||||
</style>
|
||||
<g>
|
||||
<defs>
|
||||
<rect id="SVGID_1_" width="369.5" height="67.2" />
|
||||
</defs>
|
||||
<clipPath id="SVGID_00000094609971218746498830000013677106623865950620_">
|
||||
<use xlink:href="#SVGID_1_" style="overflow:visible;" />
|
||||
</clipPath>
|
||||
<path id="Path_549" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M14.4,22.4l6.1-0.5c-0.3-2.3-1.7-4.4-3.7-5.6c-2-1.2-4.4-1.8-6.8-1.6c-2.4,0-4.6,0.8-6.5,2.2c-1.9,1.4-3,3.6-2.9,5.9
|
||||
c-0.1,2.2,1,4.3,2.9,5.5c2,1.2,4.2,2.1,6.5,2.7c3.4,1.1,5.1,2.1,5.1,3.2c0,1-0.5,1.9-1.4,2.4c-0.9,0.6-1.9,0.9-3,0.8
|
||||
c-1.1,0.1-2.1-0.2-3.1-0.7c-0.9-0.4-1.4-1.3-1.5-2.3L0,34.7c0.1,2.4,1.5,4.6,3.6,5.8c2.2,1.3,4.7,2,7.3,1.9c2.5,0,5-0.7,7.1-2.1
|
||||
c2-1.3,3.2-3.6,3.2-6c0-2.1-1-4.1-2.8-5.2c-2-1.1-4-2-6.2-2.6c-1.3-0.4-2.5-0.9-3.8-1.4c-1.1-0.6-1.7-1.4-1.7-2.2
|
||||
c0-0.9,0.4-1.8,1.1-2.4c0.7-0.6,1.6-0.9,2.5-0.8c0.9-0.1,1.8,0.2,2.6,0.6C13.8,20.7,14.3,21.5,14.4,22.4 M44.7,34.3
|
||||
c-3.2,3-8.1,2.9-11.2-0.2c-3.1-3.1-3.2-8.2,0-11.4c0,0,0,0,0,0c3.1-3.1,8.1-3.2,11.2-0.2l3.9-4.2c-1.3-1.2-2.7-2.1-4.4-2.8
|
||||
c-1.6-0.6-3.3-0.9-5-0.9c-7.6,0-13.8,6.2-13.8,13.8c0,7.6,6.2,13.8,13.8,13.8c1.7,0,3.4-0.3,5-0.9c1.6-0.6,3.1-1.6,4.4-2.8
|
||||
L44.7,34.3z M124.6,20.5h7.8v-5.4h-7.8V0l-6.1,1.2v40.4h6.1L124.6,20.5z" />
|
||||
<path id="Path_550" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M143.7,19.2L143.7,19.2l0-4.2l-6.1,1.2v3.5h0v21.9h6.1V19.2z" />
|
||||
<path id="Path_551" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M243.6,19.2L243.6,19.2l0-4.1l-6.1,1.2v3.5h0v21.9h6.1L243.6,19.2z" />
|
||||
<path id="Path_552" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M110.7,29V15.1h0V15l-6.1,1.2v3.5h0v9.6c0,4-3.2,7.2-7.2,7.2c-4,0-7.2-3.2-7.2-7.2v-1.1l-6.1,1.2c0.3,7.3,6.4,13.1,13.7,12.8
|
||||
C105.1,42,110.7,36.1,110.7,29 M84.2,19.8l6.1-1.2v-3.5l0,0V15l-6.1,1.2V19.8L84.2,19.8L84.2,19.8z" />
|
||||
<path id="Path_553" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M369.5,30.7c0.1-2.1-0.2-4.2-0.7-6.3c-0.7-2.1-1.8-4.1-3.3-5.7c-5.6-5.5-14.5-5.5-20,0c-2.6,2.5-4,6-4,9.6c-0.1,4.1,1.8,7.9,5,10.4
|
||||
c1.4,1.1,2.9,2,4.6,2.5c1.7,0.6,3.5,0.9,5.3,0.9c4.7,0.1,9.2-2,12-5.7l-4.8-3.1c-1.1,1.4-2.6,2.4-4.3,3c-1.6,0.6-3.3,0.7-5,0.5
|
||||
c-1.6-0.2-3.2-0.9-4.5-1.9c-1.3-1.1-2.3-2.5-2.7-4.2L369.5,30.7z M363.3,25.8h-16c0.5-1.7,1.6-3.1,3-4.1c1.4-1.1,3.2-1.7,5-1.7
|
||||
c1.8,0,3.5,0.6,4.9,1.6C361.7,22.6,362.8,24.1,363.3,25.8 M323.8,34.1c-3.1-3.1-3.2-8.2,0-11.4c0,0,0,0,0,0
|
||||
c3.1-3.1,8.1-3.2,11.2-0.2l3.9-4.2c-1.3-1.2-2.7-2.1-4.4-2.8c-1.6-0.6-3.3-0.9-5-0.9c-7.6,0-13.8,6.2-13.8,13.8c0,0,0,0,0,0
|
||||
c0,7.6,6.2,13.8,13.8,13.8c1.7,0,3.4-0.3,5-0.9c1.6-0.6,3.1-1.6,4.4-2.8l-3.9-4.2C331.9,37.3,326.9,37.2,323.8,34.1 M303.9,41.6
|
||||
h6.1v-14c0.1-2.8-0.7-5.6-2.2-7.9c-1.3-2-3.2-3.5-5.4-4.3c-2.1-0.9-4.4-1.1-6.7-0.7c-2.3,0.3-4.4,1.3-6.1,2.9V15l-6.1,1.2v0.4h0v25
|
||||
h6.1v-14c-0.1-2,0.8-3.9,2.3-5.3c2.8-2.3,7-2.3,9.8,0c1.5,1.3,2.4,3.3,2.3,5.3L303.9,41.6z M255.4,30.7h22.3
|
||||
c0.1-2.1-0.2-4.2-0.7-6.3c-0.7-2.1-1.8-4.1-3.3-5.7c-5.6-5.5-14.5-5.5-20,0c-2.6,2.5-4,6-4,9.6c-0.1,4.1,1.8,7.9,5,10.4
|
||||
c1.4,1.1,2.9,2,4.6,2.5c1.7,0.6,3.5,0.9,5.3,0.9c4.7,0.1,9.2-2,12-5.7l-4.8-3.1c-1.1,1.4-2.6,2.4-4.3,3c-1.6,0.6-3.3,0.7-5,0.5
|
||||
c-1.6-0.2-3.2-0.9-4.5-1.9C256.8,33.8,255.8,32.4,255.4,30.7 M255.6,25.8c0.5-1.7,1.6-3.1,3-4.1c1.4-1.1,3.2-1.7,5-1.7
|
||||
c1.8,0,3.5,0.6,4.9,1.6c1.5,1,2.6,2.4,3,4.2H255.6z M217.1,34.1c-3.1-3.1-3.2-8.2,0-11.4c0,0,0,0,0,0c3.1-3.1,8.1-3.2,11.2-0.2
|
||||
l3.9-4.2c-1.3-1.2-2.7-2.1-4.4-2.8c-1.6-0.6-3.3-0.9-5-0.9c-7.6,0-13.8,6.2-13.8,13.8c0,7.6,6.2,13.8,13.8,13.8
|
||||
c1.7,0,3.4-0.3,5-0.9c1.6-0.6,3.1-1.6,4.4-2.8l-3.9-4.2C225.2,37.3,220.2,37.2,217.1,34.1 M190.4,22.8c0-0.9,0.4-1.8,1.1-2.4
|
||||
c0.7-0.6,1.6-0.9,2.5-0.8c0.9-0.1,1.8,0.2,2.6,0.6c0.8,0.5,1.3,1.3,1.5,2.2l6.1-0.5c-0.3-2.3-1.7-4.4-3.7-5.6
|
||||
c-2-1.2-4.4-1.8-6.8-1.6c-2.4,0-4.6,0.8-6.5,2.2c-1.9,1.4-3,3.6-2.9,5.9c-0.1,2.2,1,4.3,2.9,5.5c2,1.2,4.2,2.1,6.5,2.7
|
||||
c3.4,1.1,5.1,2.1,5.1,3.2c0,1-0.5,1.9-1.4,2.4c-0.9,0.6-1.9,0.9-3,0.8c-1.1,0.1-2.1-0.2-3.1-0.7c-0.9-0.4-1.4-1.3-1.5-2.3l-6.1,0.2
|
||||
c0.1,2.4,1.5,4.6,3.6,5.8c2.2,1.3,4.7,2,7.3,1.9c2.5,0,5-0.7,7.1-2.1c2-1.3,3.2-3.6,3.2-6c0-2.1-1-4.1-2.8-5.2c-2-1.1-4-2-6.2-2.6
|
||||
c-1.3-0.4-2.5-0.9-3.8-1.4C191,24.4,190.4,23.6,190.4,22.8 M172,41.7h6.1v-14c0.1-2.8-0.7-5.6-2.2-7.9c-1.3-2-3.2-3.5-5.4-4.3
|
||||
c-2.1-0.9-4.4-1.1-6.7-0.7c-2.3,0.3-4.4,1.3-6.1,2.9V15l-6.1,1.2v0.1h0v25.3h6.1v-14c-0.1-2,0.8-3.9,2.3-5.3c2.8-2.3,7-2.3,9.8,0
|
||||
c1.5,1.3,2.4,3.3,2.3,5.3L172,41.7z" />
|
||||
<path id="Path_554" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M94.5,25.3l-1-5.2l-14.1,2.8l0.3,1.4l-1.5,0.3c-0.6-2.3-1.9-4.3-3.5-5.9c-5.4-5.4-14.1-5.4-19.5,0c-2.6,2.6-4.1,6.1-4.1,9.8
|
||||
c0,7.6,6.1,13.8,13.7,13.8c3.7,0,7.2-1.5,9.8-4.1c2.6-2.5,4.1-6.1,4.1-9.7c0-0.5,0-1-0.1-1.6l1.5-0.3l0.3,1.5L94.5,25.3z
|
||||
M72.4,28.4c0.1,1.9-0.6,3.8-1.9,5.3c-3,3.1-8.1,3.2-11.2,0.2c-0.1-0.1-0.1-0.1-0.2-0.2c-1.3-1.5-1.9-3.3-1.9-5.3
|
||||
c-0.1-1.9,0.6-3.8,1.9-5.3c1.5-1.6,3.5-2.5,5.7-2.4c2.1-0.1,4.2,0.8,5.6,2.4C71.8,24.6,72.4,26.5,72.4,28.4" />
|
||||
<path id="Path_555" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M71.2,64.6h-4L66.4,67h-1.8l3.9-10.2h1.6L73.9,67h-1.9L71.2,64.6z M67.8,63.2h3L69.2,59L67.8,63.2z" />
|
||||
|
||||
<rect id="Rectangle_1378" x="78.1" y="56.8" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" width="1.8" height="10.2" />
|
||||
|
||||
<rect id="Rectangle_1379" x="84.4" y="62" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" width="3.7" height="1.4" />
|
||||
<path id="Path_556" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="M92.5,67
|
||||
V56.8H96c0.9-0.1,1.9,0.2,2.6,0.7c0.6,0.5,1,1.3,0.9,2.1c0,0.5-0.1,0.9-0.4,1.3c-0.3,0.4-0.7,0.7-1.1,0.9c0.5,0.1,1,0.4,1.3,0.8
|
||||
c0.3,0.4,0.5,1,0.5,1.5c0.1,0.8-0.3,1.7-0.9,2.2c-0.8,0.6-1.7,0.8-2.6,0.8L92.5,67z M94.3,61.1H96c0.5,0,0.9-0.1,1.3-0.4
|
||||
c0.3-0.3,0.5-0.7,0.5-1.1c0-0.4-0.1-0.8-0.4-1.1c-0.4-0.3-0.9-0.4-1.3-0.3h-1.7V61.1z M94.3,62.4v3.2h2c0.5,0,0.9-0.1,1.3-0.4
|
||||
c0.6-0.6,0.6-1.7,0-2.3c-0.3-0.3-0.7-0.5-1.1-0.5L94.3,62.4z" />
|
||||
<path id="Path_557" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M110.2,64.6h-4l-0.8,2.4h-1.9l3.9-10.2h1.6l3.9,10.2H111L110.2,64.6z M106.7,63.2h3l-1.5-4.2L106.7,63.2z" />
|
||||
<path id="Path_558" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M122.3,64.4c0-0.4-0.2-0.8-0.5-1c-0.5-0.3-1.1-0.6-1.7-0.7c-0.7-0.2-1.3-0.5-2-0.8c-0.9-0.4-1.4-1.3-1.4-2.3c0-0.8,0.4-1.6,1-2
|
||||
c0.8-0.6,1.7-0.8,2.6-0.8c0.7,0,1.3,0.1,1.9,0.4c0.5,0.2,1,0.6,1.3,1.1c0.3,0.5,0.5,1,0.5,1.6h-1.8c0-0.5-0.2-0.9-0.5-1.3
|
||||
c-0.4-0.3-0.9-0.5-1.4-0.5c-0.5,0-1,0.1-1.4,0.4c-0.3,0.2-0.5,0.6-0.5,1c0,0.4,0.2,0.7,0.5,0.9c0.5,0.3,1.1,0.6,1.7,0.7
|
||||
c0.7,0.2,1.3,0.5,1.9,0.8c0.4,0.3,0.8,0.6,1.1,1c0.2,0.4,0.3,0.9,0.3,1.4c0,0.8-0.3,1.6-1,2c-0.8,0.5-1.7,0.8-2.7,0.8
|
||||
c-0.7,0-1.4-0.1-2-0.4c-0.6-0.2-1.1-0.6-1.4-1.1c-0.3-0.5-0.5-1.1-0.5-1.7h1.8c0,0.5,0.2,1,0.6,1.3c0.5,0.3,1.1,0.5,1.7,0.5
|
||||
c0.5,0,1-0.1,1.4-0.4C122.1,65.1,122.3,64.8,122.3,64.4" />
|
||||
<path id="Path_559" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M134.4,62.5h-4.2v3.1h4.9V67h-6.7V56.8h6.6v1.4h-4.9v2.8h4.2L134.4,62.5z" />
|
||||
<path id="Path_560" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M139.5,67V56.8h3c0.8,0,1.7,0.2,2.4,0.6c0.7,0.4,1.3,1,1.6,1.7c0.4,0.8,0.6,1.7,0.6,2.5v0.5c0,0.9-0.2,1.8-0.6,2.6
|
||||
c-0.4,0.7-0.9,1.3-1.6,1.7c-0.8,0.4-1.6,0.6-2.5,0.6L139.5,67z M141.3,58.2v7.4h1.2c0.8,0.1,1.6-0.3,2.1-0.9
|
||||
c0.5-0.7,0.8-1.6,0.8-2.5v-0.6c0.1-0.9-0.2-1.8-0.7-2.5c-0.5-0.6-1.3-0.9-2.1-0.9L141.3,58.2z" />
|
||||
<path id="Path_561" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M165.2,58.2H162V67h-1.8v-8.8h-3.2v-1.4h8.1V58.2z" />
|
||||
<path id="Path_562" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M175.4,62.5h-4.2v3.1h4.9V67h-6.7V56.8h6.6v1.4h-4.9v2.8h4.2L175.4,62.5z" />
|
||||
<path id="Path_563" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M188.2,63.7c0,1-0.5,1.9-1.2,2.6c-0.8,0.6-1.8,1-2.8,0.9c-0.8,0-1.5-0.2-2.2-0.6c-0.6-0.4-1.1-1-1.4-1.6c-0.4-0.8-0.5-1.6-0.5-2.5
|
||||
v-1c0-0.9,0.2-1.7,0.5-2.6c0.3-0.7,0.8-1.3,1.5-1.7c0.7-0.4,1.4-0.6,2.2-0.6c1,0,2,0.3,2.7,0.9c0.7,0.7,1.2,1.6,1.2,2.6h-1.8
|
||||
c0-0.6-0.2-1.2-0.6-1.6c-0.4-0.4-1-0.5-1.5-0.5c-0.7,0-1.4,0.3-1.8,0.8c-0.5,0.7-0.7,1.6-0.6,2.5v0.9c-0.1,0.9,0.2,1.8,0.6,2.5
|
||||
c0.4,0.6,1,0.9,1.7,0.9c0.6,0,1.1-0.1,1.6-0.5c0.4-0.4,0.6-1,0.7-1.6L188.2,63.7z" />
|
||||
<path id="Path_564" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M200.8,67H199v-4.5h-4.6V67h-1.8V56.8h1.8v4.3h4.6v-4.3h1.8L200.8,67z" />
|
||||
<path id="Path_565" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M219.2,58.2H216V67h-1.8v-8.8h-3.2v-1.4h8.1V58.2z" />
|
||||
<path id="Path_566" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M227.2,63.1h-2V67h-1.8V56.8h3.6c1-0.1,1.9,0.2,2.7,0.8c0.7,0.6,1,1.4,1,2.3c0,0.6-0.2,1.2-0.5,1.7c-0.4,0.5-0.8,0.8-1.4,1.1
|
||||
l2.3,4.3V67h-1.9L227.2,63.1z M225.2,61.7h1.8c0.5,0,1-0.1,1.4-0.5c0.3-0.3,0.5-0.8,0.5-1.2c0-0.5-0.1-0.9-0.5-1.3
|
||||
c-0.4-0.3-0.9-0.5-1.4-0.5h-1.9L225.2,61.7z" />
|
||||
<path id="Path_567" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M241,64.6h-4l-0.8,2.4h-1.8l3.9-10.2h1.6l3.9,10.2h-1.9L241,64.6z M237.6,63.2h3L239,59L237.6,63.2z" />
|
||||
<path id="Path_568" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M255.9,67h-1.8l-4.6-7.3V67h-1.8V56.8h1.8l4.6,7.3v-7.3h1.8L255.9,67z" />
|
||||
<path id="Path_569" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M266.2,64.4c0-0.4-0.2-0.8-0.5-1c-0.5-0.3-1.1-0.6-1.7-0.7c-0.7-0.2-1.3-0.5-2-0.8c-0.9-0.4-1.4-1.3-1.4-2.3c0-0.8,0.4-1.6,1-2
|
||||
c0.8-0.6,1.7-0.8,2.6-0.8c0.7,0,1.3,0.1,1.9,0.4c0.5,0.2,1,0.6,1.3,1.1c0.3,0.5,0.5,1,0.5,1.6h-1.8c0-0.5-0.2-0.9-0.5-1.3
|
||||
c-0.4-0.3-0.9-0.5-1.4-0.5c-0.5,0-1,0.1-1.4,0.4c-0.3,0.2-0.5,0.6-0.5,1c0,0.4,0.2,0.7,0.5,0.9c0.5,0.3,1.1,0.6,1.7,0.7
|
||||
c0.7,0.2,1.3,0.5,1.9,0.8c0.4,0.3,0.8,0.6,1.1,1c0.2,0.4,0.3,0.9,0.3,1.4c0,0.8-0.3,1.6-1,2c-0.8,0.5-1.7,0.8-2.7,0.8
|
||||
c-0.7,0-1.4-0.1-2-0.4c-0.6-0.2-1.1-0.6-1.4-1.1c-0.3-0.5-0.5-1.1-0.5-1.7h1.8c0,0.5,0.2,1,0.6,1.3c0.5,0.3,1.1,0.5,1.7,0.5
|
||||
c0.5,0,1-0.1,1.4-0.4C266,65.1,266.2,64.8,266.2,64.4" />
|
||||
<path id="Path_570" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M278.3,62.7h-4.1V67h-1.8V56.8h6.5v1.4h-4.7v3h4.1L278.3,62.7z" />
|
||||
<path id="Path_571" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M289.2,62.5H285v3.1h4.9V67h-6.7V56.8h6.6v1.4H285v2.8h4.2L289.2,62.5z" />
|
||||
<path id="Path_572" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#ffffff;" d="
|
||||
M298,63.1h-2V67h-1.8V56.8h3.6c1-0.1,1.9,0.2,2.7,0.8c0.7,0.6,1,1.4,1,2.3c0,0.6-0.2,1.2-0.5,1.7c-0.4,0.5-0.8,0.8-1.4,1.1l2.3,4.3
|
||||
V67H300L298,63.1z M296,61.7h1.8c0.5,0,1-0.1,1.4-0.5c0.3-0.3,0.5-0.8,0.5-1.2c0-0.5-0.1-0.9-0.5-1.3c-0.4-0.3-0.9-0.5-1.4-0.5H296
|
||||
L296,61.7z" />
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 12 KiB |
121
media/scoutinscience.svg
Normal file
121
media/scoutinscience.svg
Normal file
|
|
@ -0,0 +1,121 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 26.2.1, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
|
||||
<svg version="1.1" id="Group_805" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 369.5 67.2" style="enable-background:new 0 0 369.5 67.2;" xml:space="preserve">
|
||||
<style type="text/css">
|
||||
.st0{clip-path:url(#SVGID_00000084499246685778113730000005394386986623376315_);fill:#14C9C0;}
|
||||
.st1{clip-path:url(#SVGID_00000084499246685778113730000005394386986623376315_);fill:#161615;}
|
||||
</style>
|
||||
<g>
|
||||
<defs>
|
||||
<rect id="SVGID_1_" width="369.5" height="67.2" />
|
||||
</defs>
|
||||
<clipPath id="SVGID_00000094609971218746498830000013677106623865950620_">
|
||||
<use xlink:href="#SVGID_1_" style="overflow:visible;" />
|
||||
</clipPath>
|
||||
<path id="Path_549" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M14.4,22.4l6.1-0.5c-0.3-2.3-1.7-4.4-3.7-5.6c-2-1.2-4.4-1.8-6.8-1.6c-2.4,0-4.6,0.8-6.5,2.2c-1.9,1.4-3,3.6-2.9,5.9
|
||||
c-0.1,2.2,1,4.3,2.9,5.5c2,1.2,4.2,2.1,6.5,2.7c3.4,1.1,5.1,2.1,5.1,3.2c0,1-0.5,1.9-1.4,2.4c-0.9,0.6-1.9,0.9-3,0.8
|
||||
c-1.1,0.1-2.1-0.2-3.1-0.7c-0.9-0.4-1.4-1.3-1.5-2.3L0,34.7c0.1,2.4,1.5,4.6,3.6,5.8c2.2,1.3,4.7,2,7.3,1.9c2.5,0,5-0.7,7.1-2.1
|
||||
c2-1.3,3.2-3.6,3.2-6c0-2.1-1-4.1-2.8-5.2c-2-1.1-4-2-6.2-2.6c-1.3-0.4-2.5-0.9-3.8-1.4c-1.1-0.6-1.7-1.4-1.7-2.2
|
||||
c0-0.9,0.4-1.8,1.1-2.4c0.7-0.6,1.6-0.9,2.5-0.8c0.9-0.1,1.8,0.2,2.6,0.6C13.8,20.7,14.3,21.5,14.4,22.4 M44.7,34.3
|
||||
c-3.2,3-8.1,2.9-11.2-0.2c-3.1-3.1-3.2-8.2,0-11.4c0,0,0,0,0,0c3.1-3.1,8.1-3.2,11.2-0.2l3.9-4.2c-1.3-1.2-2.7-2.1-4.4-2.8
|
||||
c-1.6-0.6-3.3-0.9-5-0.9c-7.6,0-13.8,6.2-13.8,13.8c0,7.6,6.2,13.8,13.8,13.8c1.7,0,3.4-0.3,5-0.9c1.6-0.6,3.1-1.6,4.4-2.8
|
||||
L44.7,34.3z M124.6,20.5h7.8v-5.4h-7.8V0l-6.1,1.2v40.4h6.1L124.6,20.5z" />
|
||||
<path id="Path_550" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M143.7,19.2L143.7,19.2l0-4.2l-6.1,1.2v3.5h0v21.9h6.1V19.2z" />
|
||||
<path id="Path_551" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M243.6,19.2L243.6,19.2l0-4.1l-6.1,1.2v3.5h0v21.9h6.1L243.6,19.2z" />
|
||||
<path id="Path_552" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M110.7,29V15.1h0V15l-6.1,1.2v3.5h0v9.6c0,4-3.2,7.2-7.2,7.2c-4,0-7.2-3.2-7.2-7.2v-1.1l-6.1,1.2c0.3,7.3,6.4,13.1,13.7,12.8
|
||||
C105.1,42,110.7,36.1,110.7,29 M84.2,19.8l6.1-1.2v-3.5l0,0V15l-6.1,1.2V19.8L84.2,19.8L84.2,19.8z" />
|
||||
<path id="Path_553" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#14C9C0;" d="
|
||||
M369.5,30.7c0.1-2.1-0.2-4.2-0.7-6.3c-0.7-2.1-1.8-4.1-3.3-5.7c-5.6-5.5-14.5-5.5-20,0c-2.6,2.5-4,6-4,9.6c-0.1,4.1,1.8,7.9,5,10.4
|
||||
c1.4,1.1,2.9,2,4.6,2.5c1.7,0.6,3.5,0.9,5.3,0.9c4.7,0.1,9.2-2,12-5.7l-4.8-3.1c-1.1,1.4-2.6,2.4-4.3,3c-1.6,0.6-3.3,0.7-5,0.5
|
||||
c-1.6-0.2-3.2-0.9-4.5-1.9c-1.3-1.1-2.3-2.5-2.7-4.2L369.5,30.7z M363.3,25.8h-16c0.5-1.7,1.6-3.1,3-4.1c1.4-1.1,3.2-1.7,5-1.7
|
||||
c1.8,0,3.5,0.6,4.9,1.6C361.7,22.6,362.8,24.1,363.3,25.8 M323.8,34.1c-3.1-3.1-3.2-8.2,0-11.4c0,0,0,0,0,0
|
||||
c3.1-3.1,8.1-3.2,11.2-0.2l3.9-4.2c-1.3-1.2-2.7-2.1-4.4-2.8c-1.6-0.6-3.3-0.9-5-0.9c-7.6,0-13.8,6.2-13.8,13.8c0,0,0,0,0,0
|
||||
c0,7.6,6.2,13.8,13.8,13.8c1.7,0,3.4-0.3,5-0.9c1.6-0.6,3.1-1.6,4.4-2.8l-3.9-4.2C331.9,37.3,326.9,37.2,323.8,34.1 M303.9,41.6
|
||||
h6.1v-14c0.1-2.8-0.7-5.6-2.2-7.9c-1.3-2-3.2-3.5-5.4-4.3c-2.1-0.9-4.4-1.1-6.7-0.7c-2.3,0.3-4.4,1.3-6.1,2.9V15l-6.1,1.2v0.4h0v25
|
||||
h6.1v-14c-0.1-2,0.8-3.9,2.3-5.3c2.8-2.3,7-2.3,9.8,0c1.5,1.3,2.4,3.3,2.3,5.3L303.9,41.6z M255.4,30.7h22.3
|
||||
c0.1-2.1-0.2-4.2-0.7-6.3c-0.7-2.1-1.8-4.1-3.3-5.7c-5.6-5.5-14.5-5.5-20,0c-2.6,2.5-4,6-4,9.6c-0.1,4.1,1.8,7.9,5,10.4
|
||||
c1.4,1.1,2.9,2,4.6,2.5c1.7,0.6,3.5,0.9,5.3,0.9c4.7,0.1,9.2-2,12-5.7l-4.8-3.1c-1.1,1.4-2.6,2.4-4.3,3c-1.6,0.6-3.3,0.7-5,0.5
|
||||
c-1.6-0.2-3.2-0.9-4.5-1.9C256.8,33.8,255.8,32.4,255.4,30.7 M255.6,25.8c0.5-1.7,1.6-3.1,3-4.1c1.4-1.1,3.2-1.7,5-1.7
|
||||
c1.8,0,3.5,0.6,4.9,1.6c1.5,1,2.6,2.4,3,4.2H255.6z M217.1,34.1c-3.1-3.1-3.2-8.2,0-11.4c0,0,0,0,0,0c3.1-3.1,8.1-3.2,11.2-0.2
|
||||
l3.9-4.2c-1.3-1.2-2.7-2.1-4.4-2.8c-1.6-0.6-3.3-0.9-5-0.9c-7.6,0-13.8,6.2-13.8,13.8c0,7.6,6.2,13.8,13.8,13.8
|
||||
c1.7,0,3.4-0.3,5-0.9c1.6-0.6,3.1-1.6,4.4-2.8l-3.9-4.2C225.2,37.3,220.2,37.2,217.1,34.1 M190.4,22.8c0-0.9,0.4-1.8,1.1-2.4
|
||||
c0.7-0.6,1.6-0.9,2.5-0.8c0.9-0.1,1.8,0.2,2.6,0.6c0.8,0.5,1.3,1.3,1.5,2.2l6.1-0.5c-0.3-2.3-1.7-4.4-3.7-5.6
|
||||
c-2-1.2-4.4-1.8-6.8-1.6c-2.4,0-4.6,0.8-6.5,2.2c-1.9,1.4-3,3.6-2.9,5.9c-0.1,2.2,1,4.3,2.9,5.5c2,1.2,4.2,2.1,6.5,2.7
|
||||
c3.4,1.1,5.1,2.1,5.1,3.2c0,1-0.5,1.9-1.4,2.4c-0.9,0.6-1.9,0.9-3,0.8c-1.1,0.1-2.1-0.2-3.1-0.7c-0.9-0.4-1.4-1.3-1.5-2.3l-6.1,0.2
|
||||
c0.1,2.4,1.5,4.6,3.6,5.8c2.2,1.3,4.7,2,7.3,1.9c2.5,0,5-0.7,7.1-2.1c2-1.3,3.2-3.6,3.2-6c0-2.1-1-4.1-2.8-5.2c-2-1.1-4-2-6.2-2.6
|
||||
c-1.3-0.4-2.5-0.9-3.8-1.4C191,24.4,190.4,23.6,190.4,22.8 M172,41.7h6.1v-14c0.1-2.8-0.7-5.6-2.2-7.9c-1.3-2-3.2-3.5-5.4-4.3
|
||||
c-2.1-0.9-4.4-1.1-6.7-0.7c-2.3,0.3-4.4,1.3-6.1,2.9V15l-6.1,1.2v0.1h0v25.3h6.1v-14c-0.1-2,0.8-3.9,2.3-5.3c2.8-2.3,7-2.3,9.8,0
|
||||
c1.5,1.3,2.4,3.3,2.3,5.3L172,41.7z" />
|
||||
<path id="Path_554" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M94.5,25.3l-1-5.2l-14.1,2.8l0.3,1.4l-1.5,0.3c-0.6-2.3-1.9-4.3-3.5-5.9c-5.4-5.4-14.1-5.4-19.5,0c-2.6,2.6-4.1,6.1-4.1,9.8
|
||||
c0,7.6,6.1,13.8,13.7,13.8c3.7,0,7.2-1.5,9.8-4.1c2.6-2.5,4.1-6.1,4.1-9.7c0-0.5,0-1-0.1-1.6l1.5-0.3l0.3,1.5L94.5,25.3z
|
||||
M72.4,28.4c0.1,1.9-0.6,3.8-1.9,5.3c-3,3.1-8.1,3.2-11.2,0.2c-0.1-0.1-0.1-0.1-0.2-0.2c-1.3-1.5-1.9-3.3-1.9-5.3
|
||||
c-0.1-1.9,0.6-3.8,1.9-5.3c1.5-1.6,3.5-2.5,5.7-2.4c2.1-0.1,4.2,0.8,5.6,2.4C71.8,24.6,72.4,26.5,72.4,28.4" />
|
||||
<path id="Path_555" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M71.2,64.6h-4L66.4,67h-1.8l3.9-10.2h1.6L73.9,67h-1.9L71.2,64.6z M67.8,63.2h3L69.2,59L67.8,63.2z" />
|
||||
|
||||
<rect id="Rectangle_1378" x="78.1" y="56.8" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" width="1.8" height="10.2" />
|
||||
|
||||
<rect id="Rectangle_1379" x="84.4" y="62" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" width="3.7" height="1.4" />
|
||||
<path id="Path_556" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="M92.5,67
|
||||
V56.8H96c0.9-0.1,1.9,0.2,2.6,0.7c0.6,0.5,1,1.3,0.9,2.1c0,0.5-0.1,0.9-0.4,1.3c-0.3,0.4-0.7,0.7-1.1,0.9c0.5,0.1,1,0.4,1.3,0.8
|
||||
c0.3,0.4,0.5,1,0.5,1.5c0.1,0.8-0.3,1.7-0.9,2.2c-0.8,0.6-1.7,0.8-2.6,0.8L92.5,67z M94.3,61.1H96c0.5,0,0.9-0.1,1.3-0.4
|
||||
c0.3-0.3,0.5-0.7,0.5-1.1c0-0.4-0.1-0.8-0.4-1.1c-0.4-0.3-0.9-0.4-1.3-0.3h-1.7V61.1z M94.3,62.4v3.2h2c0.5,0,0.9-0.1,1.3-0.4
|
||||
c0.6-0.6,0.6-1.7,0-2.3c-0.3-0.3-0.7-0.5-1.1-0.5L94.3,62.4z" />
|
||||
<path id="Path_557" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M110.2,64.6h-4l-0.8,2.4h-1.9l3.9-10.2h1.6l3.9,10.2H111L110.2,64.6z M106.7,63.2h3l-1.5-4.2L106.7,63.2z" />
|
||||
<path id="Path_558" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M122.3,64.4c0-0.4-0.2-0.8-0.5-1c-0.5-0.3-1.1-0.6-1.7-0.7c-0.7-0.2-1.3-0.5-2-0.8c-0.9-0.4-1.4-1.3-1.4-2.3c0-0.8,0.4-1.6,1-2
|
||||
c0.8-0.6,1.7-0.8,2.6-0.8c0.7,0,1.3,0.1,1.9,0.4c0.5,0.2,1,0.6,1.3,1.1c0.3,0.5,0.5,1,0.5,1.6h-1.8c0-0.5-0.2-0.9-0.5-1.3
|
||||
c-0.4-0.3-0.9-0.5-1.4-0.5c-0.5,0-1,0.1-1.4,0.4c-0.3,0.2-0.5,0.6-0.5,1c0,0.4,0.2,0.7,0.5,0.9c0.5,0.3,1.1,0.6,1.7,0.7
|
||||
c0.7,0.2,1.3,0.5,1.9,0.8c0.4,0.3,0.8,0.6,1.1,1c0.2,0.4,0.3,0.9,0.3,1.4c0,0.8-0.3,1.6-1,2c-0.8,0.5-1.7,0.8-2.7,0.8
|
||||
c-0.7,0-1.4-0.1-2-0.4c-0.6-0.2-1.1-0.6-1.4-1.1c-0.3-0.5-0.5-1.1-0.5-1.7h1.8c0,0.5,0.2,1,0.6,1.3c0.5,0.3,1.1,0.5,1.7,0.5
|
||||
c0.5,0,1-0.1,1.4-0.4C122.1,65.1,122.3,64.8,122.3,64.4" />
|
||||
<path id="Path_559" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M134.4,62.5h-4.2v3.1h4.9V67h-6.7V56.8h6.6v1.4h-4.9v2.8h4.2L134.4,62.5z" />
|
||||
<path id="Path_560" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M139.5,67V56.8h3c0.8,0,1.7,0.2,2.4,0.6c0.7,0.4,1.3,1,1.6,1.7c0.4,0.8,0.6,1.7,0.6,2.5v0.5c0,0.9-0.2,1.8-0.6,2.6
|
||||
c-0.4,0.7-0.9,1.3-1.6,1.7c-0.8,0.4-1.6,0.6-2.5,0.6L139.5,67z M141.3,58.2v7.4h1.2c0.8,0.1,1.6-0.3,2.1-0.9
|
||||
c0.5-0.7,0.8-1.6,0.8-2.5v-0.6c0.1-0.9-0.2-1.8-0.7-2.5c-0.5-0.6-1.3-0.9-2.1-0.9L141.3,58.2z" />
|
||||
<path id="Path_561" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M165.2,58.2H162V67h-1.8v-8.8h-3.2v-1.4h8.1V58.2z" />
|
||||
<path id="Path_562" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M175.4,62.5h-4.2v3.1h4.9V67h-6.7V56.8h6.6v1.4h-4.9v2.8h4.2L175.4,62.5z" />
|
||||
<path id="Path_563" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M188.2,63.7c0,1-0.5,1.9-1.2,2.6c-0.8,0.6-1.8,1-2.8,0.9c-0.8,0-1.5-0.2-2.2-0.6c-0.6-0.4-1.1-1-1.4-1.6c-0.4-0.8-0.5-1.6-0.5-2.5
|
||||
v-1c0-0.9,0.2-1.7,0.5-2.6c0.3-0.7,0.8-1.3,1.5-1.7c0.7-0.4,1.4-0.6,2.2-0.6c1,0,2,0.3,2.7,0.9c0.7,0.7,1.2,1.6,1.2,2.6h-1.8
|
||||
c0-0.6-0.2-1.2-0.6-1.6c-0.4-0.4-1-0.5-1.5-0.5c-0.7,0-1.4,0.3-1.8,0.8c-0.5,0.7-0.7,1.6-0.6,2.5v0.9c-0.1,0.9,0.2,1.8,0.6,2.5
|
||||
c0.4,0.6,1,0.9,1.7,0.9c0.6,0,1.1-0.1,1.6-0.5c0.4-0.4,0.6-1,0.7-1.6L188.2,63.7z" />
|
||||
<path id="Path_564" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M200.8,67H199v-4.5h-4.6V67h-1.8V56.8h1.8v4.3h4.6v-4.3h1.8L200.8,67z" />
|
||||
<path id="Path_565" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M219.2,58.2H216V67h-1.8v-8.8h-3.2v-1.4h8.1V58.2z" />
|
||||
<path id="Path_566" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M227.2,63.1h-2V67h-1.8V56.8h3.6c1-0.1,1.9,0.2,2.7,0.8c0.7,0.6,1,1.4,1,2.3c0,0.6-0.2,1.2-0.5,1.7c-0.4,0.5-0.8,0.8-1.4,1.1
|
||||
l2.3,4.3V67h-1.9L227.2,63.1z M225.2,61.7h1.8c0.5,0,1-0.1,1.4-0.5c0.3-0.3,0.5-0.8,0.5-1.2c0-0.5-0.1-0.9-0.5-1.3
|
||||
c-0.4-0.3-0.9-0.5-1.4-0.5h-1.9L225.2,61.7z" />
|
||||
<path id="Path_567" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M241,64.6h-4l-0.8,2.4h-1.8l3.9-10.2h1.6l3.9,10.2h-1.9L241,64.6z M237.6,63.2h3L239,59L237.6,63.2z" />
|
||||
<path id="Path_568" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M255.9,67h-1.8l-4.6-7.3V67h-1.8V56.8h1.8l4.6,7.3v-7.3h1.8L255.9,67z" />
|
||||
<path id="Path_569" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M266.2,64.4c0-0.4-0.2-0.8-0.5-1c-0.5-0.3-1.1-0.6-1.7-0.7c-0.7-0.2-1.3-0.5-2-0.8c-0.9-0.4-1.4-1.3-1.4-2.3c0-0.8,0.4-1.6,1-2
|
||||
c0.8-0.6,1.7-0.8,2.6-0.8c0.7,0,1.3,0.1,1.9,0.4c0.5,0.2,1,0.6,1.3,1.1c0.3,0.5,0.5,1,0.5,1.6h-1.8c0-0.5-0.2-0.9-0.5-1.3
|
||||
c-0.4-0.3-0.9-0.5-1.4-0.5c-0.5,0-1,0.1-1.4,0.4c-0.3,0.2-0.5,0.6-0.5,1c0,0.4,0.2,0.7,0.5,0.9c0.5,0.3,1.1,0.6,1.7,0.7
|
||||
c0.7,0.2,1.3,0.5,1.9,0.8c0.4,0.3,0.8,0.6,1.1,1c0.2,0.4,0.3,0.9,0.3,1.4c0,0.8-0.3,1.6-1,2c-0.8,0.5-1.7,0.8-2.7,0.8
|
||||
c-0.7,0-1.4-0.1-2-0.4c-0.6-0.2-1.1-0.6-1.4-1.1c-0.3-0.5-0.5-1.1-0.5-1.7h1.8c0,0.5,0.2,1,0.6,1.3c0.5,0.3,1.1,0.5,1.7,0.5
|
||||
c0.5,0,1-0.1,1.4-0.4C266,65.1,266.2,64.8,266.2,64.4" />
|
||||
<path id="Path_570" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M278.3,62.7h-4.1V67h-1.8V56.8h6.5v1.4h-4.7v3h4.1L278.3,62.7z" />
|
||||
<path id="Path_571" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M289.2,62.5H285v3.1h4.9V67h-6.7V56.8h6.6v1.4H285v2.8h4.2L289.2,62.5z" />
|
||||
<path id="Path_572" style="clip-path:url(#SVGID_00000094609971218746498830000013677106623865950620_);fill:#161615;" d="
|
||||
M298,63.1h-2V67h-1.8V56.8h3.6c1-0.1,1.9,0.2,2.7,0.8c0.7,0.6,1,1.4,1,2.3c0,0.6-0.2,1.2-0.5,1.7c-0.4,0.5-0.8,0.8-1.4,1.1l2.3,4.3
|
||||
V67H300L298,63.1z M296,61.7h1.8c0.5,0,1-0.1,1.4-0.5c0.3-0.3,0.5-0.8,0.5-1.2c0-0.5-0.1-0.9-0.5-1.3c-0.4-0.3-0.9-0.5-1.4-0.5H296
|
||||
L296,61.7z" />
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 12 KiB |
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../assets/images/favicon.png">
|
||||
<link rel="icon" href="../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -249,13 +249,74 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -264,13 +325,107 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
<a href="../simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -302,21 +457,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -521,7 +661,7 @@ In this case, the latest version is already in the local cache, no download is r
|
|||
<a id="__codelineno-5-5" name="__codelineno-5-5" href="#__codelineno-5-5"></a><span class="na">endpoint_url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">this is optional, for backblaze, use this: https://s3.us-west-002.backblazeb2.com</span><span class="w"></span>
|
||||
</code></pre></div>
|
||||
<blockquote>
|
||||
<p>Just like in <a href="../example_secrets.ini">example secrets</a>.</p>
|
||||
<p>Just like in <a href="example_secrets.ini">example secrets</a>.</p>
|
||||
</blockquote>
|
||||
<h4 id="print-the-expected-options">Print the expected options<a class="headerlink" href="#print-the-expected-options" title="Permanent link">#</a></h4>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a>python3 -m large_file --help
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../assets/images/favicon.png">
|
||||
<link rel="icon" href="../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -249,13 +249,74 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -264,13 +325,107 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
<a href="../simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -346,21 +501,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -615,7 +755,7 @@
|
|||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
<a href="../how-to-guides/" class="md-footer__link md-footer__link--prev" aria-label="Previous: How-To Guides" rel="prev">
|
||||
<a href="../simple/deploy/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Train a domain classifier on the semantic scholar dataset" rel="prev">
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</div>
|
||||
|
|
@ -624,7 +764,7 @@
|
|||
<span class="md-footer__direction">
|
||||
Previous
|
||||
</span>
|
||||
How-To Guides
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</div>
|
||||
</div>
|
||||
</a>
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../../assets/images/favicon.png">
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -249,13 +249,74 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -264,13 +325,107 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
<a href="../../simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -302,31 +457,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -2430,27 +2560,6 @@ class="
|
|||
|
||||
<footer class="md-footer">
|
||||
|
||||
|
||||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
<a href="../../explanation/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Explanation" rel="prev">
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</div>
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Previous
|
||||
</span>
|
||||
Explanation
|
||||
</div>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
|
||||
</nav>
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../../../assets/images/favicon.png">
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -21,10 +21,10 @@
|
|||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
|
||||
|
||||
|
||||
|
|
@ -40,9 +40,9 @@
|
|||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/_mkdocstrings.css">
|
||||
<link rel="stylesheet" href="../../assets/_mkdocstrings.css">
|
||||
|
||||
<script>__md_scope=new URL("../../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
<script>__md_scope=new URL("../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
|
||||
|
||||
|
||||
|
|
@ -84,7 +84,7 @@
|
|||
|
||||
<header class="md-header" data-md-component="header">
|
||||
<nav class="md-header__inner md-grid" aria-label="Header">
|
||||
<a href="../../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
<a href="../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
|
@ -204,7 +204,7 @@
|
|||
|
||||
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
|
||||
<label class="md-nav__title" for="__drawer">
|
||||
<a href="../../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
<a href="../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
|
@ -235,7 +235,7 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../.." class="md-nav__link">
|
||||
<a href="../.." class="md-nav__link">
|
||||
Overview
|
||||
</a>
|
||||
</li>
|
||||
|
|
@ -249,13 +249,262 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" checked>
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" checked>
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link md-nav__link--active" for="__toc">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Table of contents
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#extract" class="md-nav__link">
|
||||
Extract
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#transform" class="md-nav__link">
|
||||
Transform
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#load" class="md-nav__link">
|
||||
Load
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Load">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#production-ready-backend" class="md-nav__link">
|
||||
Production-ready backend
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#next-part-2" class="md-nav__link">
|
||||
Next: Part 2
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -265,22 +514,7 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../reference/" class="md-nav__link">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
</a>
|
||||
</li>
|
||||
|
|
@ -295,28 +529,13 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../explanation/" class="md-nav__link">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -403,7 +622,7 @@
|
|||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/great_ai_example-main/src/data.ipynb" title="Edit this page" class="md-content__button md-icon">
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/simple/data.ipynb" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
|
@ -1907,7 +2126,7 @@ add_ground_truth(X, y, train_split_ratio=0.8, test_split_ratio=0.2)</div>
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -1924,6 +2143,42 @@ add_ground_truth(X, y, train_split_ratio=0.8, test_split_ratio=0.2)</div>
|
|||
|
||||
<footer class="md-footer">
|
||||
|
||||
|
||||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
<a href="../../tutorial/deploy/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Harden and deploy your app" rel="prev">
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</div>
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Previous
|
||||
</span>
|
||||
Harden and deploy your app
|
||||
</div>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="../train/" class="md-footer__link md-footer__link--next" aria-label="Next: Train a domain classifier on the semantic scholar dataset" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
</nav>
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
|
|
@ -1944,10 +2199,10 @@ add_ground_truth(X, y, train_split_ratio=0.8, test_split_ratio=0.2)</div>
|
|||
<div class="md-dialog" data-md-component="dialog">
|
||||
<div class="md-dialog__inner md-typeset"></div>
|
||||
</div>
|
||||
<script id="__config" type="application/json">{"base": "../../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
<script id="__config" type="application/json">{"base": "../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
|
||||
|
||||
<script src="../../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
<script src="../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
|
|
@ -185,7 +185,10 @@
|
|||
" X = [d.input for d in data]\n",
|
||||
" y_actual = [d.feedback for d in data]\n",
|
||||
"\n",
|
||||
" y_predicted = [d.output.labels[0].label for d in predict_domain.process_batch(X, do_not_persist_traces=True)]\n",
|
||||
" y_predicted = [\n",
|
||||
" d.output.labels[0].label\n",
|
||||
" for d in predict_domain.process_batch(X, do_not_persist_traces=True)\n",
|
||||
" ]\n",
|
||||
" y_actual_aligned = [p if p in a else a[0] for p, a in zip(y_predicted, y_actual)]\n",
|
||||
"\n",
|
||||
" import matplotlib.pyplot as plt\n",
|
||||
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../../../assets/images/favicon.png">
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -21,10 +21,10 @@
|
|||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
|
||||
|
||||
|
||||
|
|
@ -40,9 +40,9 @@
|
|||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/_mkdocstrings.css">
|
||||
<link rel="stylesheet" href="../../assets/_mkdocstrings.css">
|
||||
|
||||
<script>__md_scope=new URL("../../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
<script>__md_scope=new URL("../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
|
||||
|
||||
|
||||
|
|
@ -84,7 +84,7 @@
|
|||
|
||||
<header class="md-header" data-md-component="header">
|
||||
<nav class="md-header__inner md-grid" aria-label="Header">
|
||||
<a href="../../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
<a href="../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
|
@ -204,7 +204,7 @@
|
|||
|
||||
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
|
||||
<label class="md-nav__title" for="__drawer">
|
||||
<a href="../../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
<a href="../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
|
@ -235,7 +235,7 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../.." class="md-nav__link">
|
||||
<a href="../.." class="md-nav__link">
|
||||
Overview
|
||||
</a>
|
||||
</li>
|
||||
|
|
@ -249,13 +249,228 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" checked>
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" checked>
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link md-nav__link--active" for="__toc">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Table of contents
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#check-accuracy-on-the-test-split" class="md-nav__link">
|
||||
Check accuracy on the test split
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -265,22 +480,7 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../reference/" class="md-nav__link">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
</a>
|
||||
</li>
|
||||
|
|
@ -295,28 +495,13 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../explanation/" class="md-nav__link">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -369,7 +554,7 @@
|
|||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/great_ai_example-main/src/deploy.ipynb" title="Edit this page" class="md-content__button md-icon">
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/simple/deploy.ipynb" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
|
@ -1666,7 +1851,10 @@ def get_label(
|
|||
<span class="n">X</span> <span class="o">=</span> <span class="p">[</span><span class="n">d</span><span class="o">.</span><span class="n">input</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">data</span><span class="p">]</span>
|
||||
<span class="n">y_actual</span> <span class="o">=</span> <span class="p">[</span><span class="n">d</span><span class="o">.</span><span class="n">feedback</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">data</span><span class="p">]</span>
|
||||
|
||||
<span class="n">y_predicted</span> <span class="o">=</span> <span class="p">[</span><span class="n">d</span><span class="o">.</span><span class="n">output</span><span class="o">.</span><span class="n">labels</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">label</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">predict_domain</span><span class="o">.</span><span class="n">process_batch</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">do_not_persist_traces</span><span class="o">=</span><span class="kc">True</span><span class="p">)]</span>
|
||||
<span class="n">y_predicted</span> <span class="o">=</span> <span class="p">[</span>
|
||||
<span class="n">d</span><span class="o">.</span><span class="n">output</span><span class="o">.</span><span class="n">labels</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">label</span>
|
||||
<span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">predict_domain</span><span class="o">.</span><span class="n">process_batch</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">do_not_persist_traces</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
<span class="p">]</span>
|
||||
<span class="n">y_actual_aligned</span> <span class="o">=</span> <span class="p">[</span><span class="n">p</span> <span class="k">if</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">a</span> <span class="k">else</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">p</span><span class="p">,</span> <span class="n">a</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">y_predicted</span><span class="p">,</span> <span class="n">y_actual</span><span class="p">)]</span>
|
||||
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
|
|
@ -1697,7 +1885,10 @@ def get_label(
|
|||
X = [d.input for d in data]
|
||||
y_actual = [d.feedback for d in data]
|
||||
|
||||
y_predicted = [d.output.labels[0].label for d in predict_domain.process_batch(X, do_not_persist_traces=True)]
|
||||
y_predicted = [
|
||||
d.output.labels[0].label
|
||||
for d in predict_domain.process_batch(X, do_not_persist_traces=True)
|
||||
]
|
||||
y_actual_aligned = [p if p in a else a[0] for p, a in zip(y_predicted, y_actual)]
|
||||
|
||||
import matplotlib.pyplot as plt
|
||||
|
|
@ -1814,7 +2005,7 @@ class="
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -1831,6 +2022,42 @@ class="
|
|||
|
||||
<footer class="md-footer">
|
||||
|
||||
|
||||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
<a href="../train/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Train a domain classifier on the semantic scholar dataset" rel="prev">
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</div>
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Previous
|
||||
</span>
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</div>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="../../reference/" class="md-footer__link md-footer__link--next" aria-label="Next: Reference" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
Reference
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
</nav>
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
|
|
@ -1851,10 +2078,10 @@ class="
|
|||
<div class="md-dialog" data-md-component="dialog">
|
||||
<div class="md-dialog__inner md-typeset"></div>
|
||||
</div>
|
||||
<script id="__config" type="application/json">{"base": "../../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
<script id="__config" type="application/json">{"base": "../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
|
||||
|
||||
<script src="../../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
<script src="../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
|
|
@ -12,7 +12,7 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../../../assets/images/favicon.png">
|
||||
<link rel="icon" href="../../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
|
@ -21,10 +21,10 @@
|
|||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/main.1d29e8d0.min.css">
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
<link rel="stylesheet" href="../../assets/stylesheets/palette.cbb835fc.min.css">
|
||||
|
||||
|
||||
|
||||
|
|
@ -40,9 +40,9 @@
|
|||
|
||||
|
||||
|
||||
<link rel="stylesheet" href="../../../assets/_mkdocstrings.css">
|
||||
<link rel="stylesheet" href="../../assets/_mkdocstrings.css">
|
||||
|
||||
<script>__md_scope=new URL("../../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
<script>__md_scope=new URL("../..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
|
||||
|
||||
|
||||
|
||||
|
|
@ -84,7 +84,7 @@
|
|||
|
||||
<header class="md-header" data-md-component="header">
|
||||
<nav class="md-header__inner md-grid" aria-label="Header">
|
||||
<a href="../../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
<a href="../.." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
|
@ -204,7 +204,7 @@
|
|||
|
||||
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
|
||||
<label class="md-nav__title" for="__drawer">
|
||||
<a href="../../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
<a href="../.." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
|
||||
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
|
||||
|
|
@ -235,7 +235,7 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../.." class="md-nav__link">
|
||||
<a href="../.." class="md-nav__link">
|
||||
Overview
|
||||
</a>
|
||||
</li>
|
||||
|
|
@ -249,13 +249,255 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../tutorials/" class="md-nav__link">
|
||||
Tutorials
|
||||
<a href="../../tutorial/" class="md-nav__link">
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../tutorial/deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" checked>
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" checked>
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link md-nav__link--active" for="__toc">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Table of contents
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#load-data-that-has-been-extracted-in-part-1" class="md-nav__link">
|
||||
Load data that has been extracted in part 1
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#optimise-and-train-multinomial-naive-bayes-classifier" class="md-nav__link">
|
||||
Optimise and train Multinomial Naive Bayes classifier
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#export-the-model-using-greatai" class="md-nav__link">
|
||||
Export the model using GreatAI
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Export the model using GreatAI">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#next-part-3" class="md-nav__link">
|
||||
Next: Part 3
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -265,22 +507,7 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../reference/" class="md-nav__link">
|
||||
<a href="../../reference/" class="md-nav__link">
|
||||
Reference
|
||||
</a>
|
||||
</li>
|
||||
|
|
@ -295,28 +522,13 @@
|
|||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../explanation/" class="md-nav__link">
|
||||
<a href="../../explanation/" class="md-nav__link">
|
||||
Explanation
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../../../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -396,7 +608,7 @@
|
|||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/great_ai_example-main/src/train.ipynb" title="Edit this page" class="md-content__button md-icon">
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/simple/train.ipynb" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
|
@ -2427,7 +2639,7 @@ save_model(classifier, key="small-domain-prediction", keep_last_n=5)</div>
|
|||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -2444,6 +2656,42 @@ save_model(classifier, key="small-domain-prediction", keep_last_n=5)</div>
|
|||
|
||||
<footer class="md-footer">
|
||||
|
||||
|
||||
<nav class="md-footer__inner md-grid" aria-label="Footer" >
|
||||
|
||||
|
||||
<a href="../data/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Train a domain classifier on the semantic scholar dataset" rel="prev">
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
|
||||
</div>
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Previous
|
||||
</span>
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</div>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<a href="../deploy/" class="md-footer__link md-footer__link--next" aria-label="Next: Train a domain classifier on the semantic scholar dataset" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
|
||||
</div>
|
||||
</a>
|
||||
|
||||
</nav>
|
||||
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-copyright">
|
||||
|
|
@ -2464,10 +2712,10 @@ save_model(classifier, key="small-domain-prediction", keep_last_n=5)</div>
|
|||
<div class="md-dialog" data-md-component="dialog">
|
||||
<div class="md-dialog__inner md-typeset"></div>
|
||||
</div>
|
||||
<script id="__config" type="application/json">{"base": "../../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
<script id="__config" type="application/json">{"base": "../..", "features": ["content.code.annotate", "content.tooltips"], "search": "../../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
|
||||
|
||||
|
||||
<script src="../../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
<script src="../../assets/javascripts/bundle.6c7ad80a.min.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
26
sitemap.xml
26
sitemap.xml
|
|
@ -2,67 +2,67 @@
|
|||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2022-07-08</lastmod>
|
||||
<lastmod>2022-07-09</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
</urlset>
|
||||
BIN
sitemap.xml.gz
BIN
sitemap.xml.gz
Binary file not shown.
214
tutorial/deploy/deploy.ipynb
Normal file
214
tutorial/deploy/deploy.ipynb
Normal file
|
|
@ -0,0 +1,214 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Harden and deploy your app\n",
|
||||
"\n",
|
||||
"Finally, it's time to deploy your model. But before you have to make sure you follow AI deployment [best-practices](https://se-ml.github.io/). In the past, this step was too often either the source of unexpected struggles, or worse, simply ignored.\n",
|
||||
"\n",
|
||||
"With `GreatAI`, it has become a matter of 2 lines of code."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;226m2022-07-09 22:38:56 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:38:56 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:38:56 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:38:56 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | GreatAI (v0.1.0): configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | 🔩 tracing_database: ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | 🔩 large_file_implementation: LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | 🔩 dashboard_table_size: 20\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:38:56 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:38:56 | WARNING | > Find out more at https://se-ml.github.io/practices\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | Fetching cached versions of my-domain-predictor\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | Latest version of my-domain-predictor is 5 (from versions: 0, 1, 2, 3, 4, 5)\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:56 | INFO | File my-domain-predictor-5 found in cache\u001b[0m\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from great_ai import GreatAI, use_model\n",
|
||||
"from great_ai.utilities import clean\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@GreatAI.create\n",
|
||||
"@use_model(\"my-domain-predictor\")\n",
|
||||
"def predict_domain(sentence, model):\n",
|
||||
" inputs = [clean(sentence)]\n",
|
||||
" return str(model.predict(inputs)[0])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Trace[str]({ 'created': '2022-07-09T20:38:56.394746',\n",
|
||||
" 'exception': None,\n",
|
||||
" 'feedback': None,\n",
|
||||
" 'logged_values': { 'arg:sentence:length': 29,\n",
|
||||
" 'arg:sentence:value': 'Mountains are just big rocks.'},\n",
|
||||
" 'models': [{'key': 'my-domain-predictor', 'version': 5}],\n",
|
||||
" 'original_execution_time_ms': 4.999,\n",
|
||||
" 'output': 'geography',\n",
|
||||
" 'tags': ['predict_domain', 'online', 'development'],\n",
|
||||
" 'trace_id': 'aad1f83d-a81f-4b8b-898e-d02f8076616f'})"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"predict_domain(\"Mountains are just big rocks.\")\n",
|
||||
"# the original return value is under the 'output' key"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[38;5;39m2022-07-09 22:38:58 | INFO | Converting notebook to Python script\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:58 | INFO | Found `predict_domain` to be the GreatAI app \u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:38:58 | INFO | Uvicorn running on http://0.0.0.0:6060 (Press CTRL+C to quit)\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:39:00 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:39:00 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:39:00 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:39:00 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | GreatAI (v0.1.0): configured ✅\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | 🔩 tracing_database: ParallelTinyDbDriver\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | 🔩 large_file_implementation: LargeFileLocal\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | 🔩 is_production: False\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | 🔩 dashboard_table_size: 20\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:39:00 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
|
||||
"\u001b[38;5;226m2022-07-09 22:39:00 | WARNING | > Find out more at https://se-ml.github.io/practices\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | Fetching cached versions of my-domain-predictor\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | Latest version of my-domain-predictor is 5 (from versions: 0, 1, 2, 3, 4, 5)\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | File my-domain-predictor-5 found in cache\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | Started server process [882179]\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | Waiting for application startup.\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:00 | INFO | Application startup complete.\u001b[0m\n",
|
||||
"^C\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:04 | INFO | Shutting down\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:04 | INFO | Waiting for application shutdown.\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:04 | INFO | Application shutdown complete.\u001b[0m\n",
|
||||
"\u001b[38;5;39m2022-07-09 22:39:04 | INFO | Finished server process [882179]\u001b[0m\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!great-ai deploy.ipynb\n",
|
||||
"# leave this running and open http://127.0.0.1:6060"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now that you've made sure your application is hardened enough for the intended use case, it is time to deploy it. The responsibilities of GreatAI end when it wraps your inference function and model into a production-ready service. You're given the freedom and responsibility to deploy this service. Fortunately, you (or your organisation) probably already has an established routine for deploying services.\n",
|
||||
"\n",
|
||||
"There are three main approaches to deploy a GreatAI service.\n",
|
||||
"\n",
|
||||
"### Manual deployment\n",
|
||||
"\n",
|
||||
"Simply run `ENVIRONMENT=production great-ai deploy.ipynb` in the command-line of a production machine.\n",
|
||||
"> This is the crudest approach, however, it might be fitting for some contexts.\n",
|
||||
"\n",
|
||||
"### Containerised deployment\n",
|
||||
"\n",
|
||||
"Run the notebook directly in a container or create a service for it using your favourite orchestrator.\n",
|
||||
"\n",
|
||||
"```sh\n",
|
||||
"docker run \\\n",
|
||||
" -p 6060:6060 \\\n",
|
||||
" --volume `pwd`:/app \\\n",
|
||||
" --rm \\\n",
|
||||
" schmelczera/great-ai deploy.ipynb\n",
|
||||
"```\n",
|
||||
"> You can replace ``pwd`` with the path to your code's folder.\n",
|
||||
"\n",
|
||||
"#### Use a Platform-as-a-Service\n",
|
||||
"\n",
|
||||
"Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers that take a Docker image as a source and handle the rest of the deployment.\n",
|
||||
"\n",
|
||||
"To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as pytorch or tensorflow.\n",
|
||||
"\n",
|
||||
"```Dockerfile\n",
|
||||
"FROM schmelczera/great-ai:latest\n",
|
||||
"\n",
|
||||
"# Remove this block if you don't have a requirements.txt\n",
|
||||
"COPY requirements.txt ./ \n",
|
||||
"RUN pip install --no-cache-dir --requirement requirements.txt\n",
|
||||
"\n",
|
||||
"# If you store your models in S3 or GridFS, it may be a \n",
|
||||
"# good idea to # cache them in the image so that you don't\n",
|
||||
"# have to download it each time a container starts\n",
|
||||
"# RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor\n",
|
||||
"\n",
|
||||
"COPY . .\n",
|
||||
"\n",
|
||||
"CMD [\"deploy.ipynb\"]\n",
|
||||
"\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### [Go back to the summary](/tutorial)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.10.4 ('.env': venv)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.4"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "02dd6d3afbfa9fbbe1037d64ad9014965528a1ccad21929d6e72f466389a68ad"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
2023
tutorial/deploy/index.html
Normal file
2023
tutorial/deploy/index.html
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -12,12 +12,12 @@
|
|||
<meta name="author" content="András Schmelczer">
|
||||
|
||||
|
||||
<link rel="icon" href="../assets/images/favicon.png">
|
||||
<link rel="icon" href="../media/favicon.ico">
|
||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
||||
|
||||
|
||||
|
||||
<title>Tutorials - GreatAI documentation</title>
|
||||
<title>Train and deploy a SOTA model - GreatAI documentation</title>
|
||||
|
||||
|
||||
|
||||
|
|
@ -69,6 +69,11 @@
|
|||
<label class="md-overlay" for="__drawer"></label>
|
||||
<div data-md-component="skip">
|
||||
|
||||
|
||||
<a href="#train-and-deploy-a-sota-model" class="md-skip">
|
||||
Skip to content
|
||||
</a>
|
||||
|
||||
</div>
|
||||
<div data-md-component="announce">
|
||||
|
||||
|
|
@ -98,7 +103,7 @@
|
|||
<div class="md-header__topic" data-md-component="header-topic">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Tutorials
|
||||
Train and deploy a SOTA model
|
||||
|
||||
</span>
|
||||
</div>
|
||||
|
|
@ -246,16 +251,146 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" checked>
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_2">
|
||||
Tutorial
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_2">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Tutorial
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--active">
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link md-nav__link--active" for="__toc">
|
||||
Train and deploy a SOTA model
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<a href="./" class="md-nav__link md-nav__link--active">
|
||||
Tutorials
|
||||
Train and deploy a SOTA model
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Table of contents
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#overview" class="md-nav__link">
|
||||
Overview
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#objectives" class="md-nav__link">
|
||||
Objectives
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#summary" class="md-nav__link">
|
||||
Summary
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Summary">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#the-training-notebook" class="md-nav__link">
|
||||
The training notebook
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#the-deployment-notebook" class="md-nav__link">
|
||||
The deployment notebook
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="train/" class="md-nav__link">
|
||||
Train your model
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="deploy/" class="md-nav__link">
|
||||
Harden and deploy your app
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
|
@ -267,13 +402,107 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3">
|
||||
User Guides
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
|
||||
<label class="md-nav__title" for="__nav_3">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
User Guides
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item md-nav__item--nested">
|
||||
|
||||
|
||||
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__link" for="__nav_3_1">
|
||||
Simple AI lifecycle with classification
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
</label>
|
||||
|
||||
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
|
||||
<label class="md-nav__title" for="__nav_3_1">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Simple AI lifecycle with classification
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../how-to-guides/" class="md-nav__link">
|
||||
How-To Guides
|
||||
<a href="../simple/data/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/train/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple/deploy/" class="md-nav__link">
|
||||
Train a domain classifier on the semantic scholar dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -305,21 +534,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="../simple-mag/train/" class="md-nav__link">
|
||||
Train a field of study (domain) classification of sentences on the MAG dataset
|
||||
</a>
|
||||
</li>
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -337,6 +551,57 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<label class="md-nav__title" for="__toc">
|
||||
<span class="md-nav__icon md-icon"></span>
|
||||
Table of contents
|
||||
</label>
|
||||
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#overview" class="md-nav__link">
|
||||
Overview
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#objectives" class="md-nav__link">
|
||||
Objectives
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#summary" class="md-nav__link">
|
||||
Summary
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Summary">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#the-training-notebook" class="md-nav__link">
|
||||
The training notebook
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#the-deployment-notebook" class="md-nav__link">
|
||||
The deployment notebook
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -351,22 +616,75 @@
|
|||
|
||||
|
||||
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/tutorials.md" title="Edit this page" class="md-content__button md-icon">
|
||||
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/tutorial/index.md" title="Edit this page" class="md-content__button md-icon">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
|
||||
</a>
|
||||
|
||||
|
||||
|
||||
<h1>Tutorials</h1>
|
||||
|
||||
|
||||
<h1 id="train-and-deploy-a-sota-model">Train and deploy a SOTA model<a class="headerlink" href="#train-and-deploy-a-sota-model" title="Permanent link">#</a></h1>
|
||||
<h2 id="overview">Overview<a class="headerlink" href="#overview" title="Permanent link">#</a></h2>
|
||||
<p>You are going to train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the <a href="https://arxiv.org/abs/1903.10676">SciBERT paper</a> in which SciBERT <a href="https://paperswithcode.com/sota/sentence-classification-on-paper-field">achieved an F1-score of 0.6571</a>. We are going to outperform it using a trivial text classification model: a <a href="https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html">Linear SVM</a>.</p>
|
||||
<p>We use the same synthetic dataset derived from the <a href="https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/">Microsoft Academic Graph</a>. The dataset is <a href="https://github.com/allenai/scibert/tree/master/data/text_classification/mag">available here</a>.</p>
|
||||
<h2 id="objectives">Objectives<a class="headerlink" href="#objectives" title="Permanent link">#</a></h2>
|
||||
<ol>
|
||||
<li>You will see how the <code>great_ai.utilities</code> can integrate into your Data Science workflow.</li>
|
||||
<li>You will use <code>great_ai.large_file</code> to version and store your trained model</li>
|
||||
<li>You will use <code>GreatAI</code> to prepare your model for a robust and responsible deployment</li>
|
||||
</ol>
|
||||
<div class="admonition success">
|
||||
<p class="admonition-title">Success</p>
|
||||
<p>You are ready to start the tutorial. Feel free to come back to the <a href="#summary">summary</a> section once you're finished.</p>
|
||||
</div>
|
||||
<div style="display: flex; justify-content: space-evenly;">
|
||||
<p><a class="md-button md-button--primary" href="train/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M160 80c0-26.51 21.5-48 48-48h32c26.5 0 48 21.49 48 48v352c0 26.5-21.5 48-48 48h-32c-26.5 0-48-21.5-48-48V80zM0 272c0-26.5 21.49-48 48-48h32c26.5 0 48 21.5 48 48v160c0 26.5-21.5 48-48 48H48c-26.51 0-48-21.5-48-48V272zM400 96c26.5 0 48 21.5 48 48v288c0 26.5-21.5 48-48 48h-32c-26.5 0-48-21.5-48-48V144c0-26.5 21.5-48 48-48h32z"/></svg></span> Train it</a></p>
|
||||
<p><a class="md-button md-button--secondary" href="deploy/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M6 20a6 6 0 0 1-6-6c0-3.09 2.34-5.64 5.35-5.96A7.496 7.496 0 0 1 12 4a7.5 7.5 0 0 1 7.35 6A5.02 5.02 0 0 1 24 15a5 5 0 0 1-5 5H6M9.09 8.4 4.5 13l4.59 4.6 1.41-1.42L7.32 13l3.18-3.18L9.09 8.4m5.82 0L13.5 9.82 16.68 13l-3.18 3.18 1.41 1.42L19.5 13l-4.59-4.6Z"/></svg></span> Deploy it</a></p>
|
||||
</div>
|
||||
<h2 id="summary">Summary<a class="headerlink" href="#summary" title="Permanent link">#</a></h2>
|
||||
<h3 id="the-training-notebook">The <a href="train/">training notebook</a><a class="headerlink" href="#the-training-notebook" title="Permanent link">#</a></h3>
|
||||
<p>We load and preprocess the dataset while relying on <code>great_ai.utilities.clean</code> for the heavy-lifting. Additionally, the preprocessing is parallelised using <code>great_ai.utilities.simple_parallel_map</code>.</p>
|
||||
<p>After training and evaluating a model, it is exported using <code>great_ai.save_model</code>.</p>
|
||||
<details class="tip">
|
||||
<summary>Using the cloud</summary>
|
||||
<p>To store your model remotely, you need to set your credentials before calling <code>save_model</code>.</p>
|
||||
<p>For example, to use <a href="https://aws.amazon.com/s3/">AWS S3</a>:
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="kn">from</span> <span class="nn">great_ai.large_file</span> <span class="kn">import</span> <span class="n">LargeFileS3</span>
|
||||
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a>
|
||||
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a><span class="n">LargeFileS3</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a> <span class="n">aws_region_name</span><span class="o">=</span><span class="s1">'eu-west-2'</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a> <span class="n">aws_access_key_id</span><span class="o">=</span><span class="s1">'MY_AWS_ACCESS_KEY'</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-6" name="__codelineno-0-6" href="#__codelineno-0-6"></a> <span class="n">aws_secret_access_key</span><span class="o">=</span><span class="s1">'MY_AWS_SECRET_KEY'</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-7" name="__codelineno-0-7" href="#__codelineno-0-7"></a> <span class="n">large_files_bucket_name</span><span class="o">=</span><span class="s1">'my_bucket_for_models'</span>
|
||||
<a id="__codelineno-0-8" name="__codelineno-0-8" href="#__codelineno-0-8"></a><span class="p">)</span>
|
||||
<a id="__codelineno-0-9" name="__codelineno-0-9" href="#__codelineno-0-9"></a>
|
||||
<a id="__codelineno-0-10" name="__codelineno-0-10" href="#__codelineno-0-10"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">save_model</span>
|
||||
<a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a>
|
||||
<a id="__codelineno-0-12" name="__codelineno-0-12" href="#__codelineno-0-12"></a><span class="n">save_model</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="s1">'my-domain-predictor'</span><span class="p">)</span>
|
||||
</code></pre></div></p>
|
||||
</details>
|
||||
<h3 id="the-deployment-notebook">The <a href="deploy/">deployment notebook</a><a class="headerlink" href="#the-deployment-notebook" title="Permanent link">#</a></h3>
|
||||
<p>We create an inference function that can be hardened by wrapping it in a <code>great_ai.GreatAI</code> instance.</p>
|
||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a><span class="kn">from</span> <span class="nn">great_ai</span> <span class="kn">import</span> <span class="n">GreatAI</span><span class="p">,</span> <span class="n">use_model</span>
|
||||
<a id="__codelineno-1-2" name="__codelineno-1-2" href="#__codelineno-1-2"></a><span class="kn">from</span> <span class="nn">great_ai.utilities</span> <span class="kn">import</span> <span class="n">clean</span>
|
||||
<a id="__codelineno-1-3" name="__codelineno-1-3" href="#__codelineno-1-3"></a>
|
||||
<a id="__codelineno-1-4" name="__codelineno-1-4" href="#__codelineno-1-4"></a><span class="nd">@GreatAI</span><span class="o">.</span><span class="n">create</span>
|
||||
<a id="__codelineno-1-5" name="__codelineno-1-5" href="#__codelineno-1-5"></a><span class="nd">@use_model</span><span class="p">(</span><span class="s1">'my-domain-predictor'</span><span class="p">)</span> <span class="c1">#(1)</span>
|
||||
<a id="__codelineno-1-6" name="__codelineno-1-6" href="#__codelineno-1-6"></a><span class="k">def</span> <span class="nf">predict_domain</span><span class="p">(</span><span class="n">sentence</span><span class="p">,</span> <span class="n">model</span><span class="p">):</span>
|
||||
<a id="__codelineno-1-7" name="__codelineno-1-7" href="#__codelineno-1-7"></a> <span class="n">inputs</span> <span class="o">=</span> <span class="p">[</span><span class="n">clean</span><span class="p">(</span><span class="n">sentence</span><span class="p">)]</span>
|
||||
<a id="__codelineno-1-8" name="__codelineno-1-8" href="#__codelineno-1-8"></a> <span class="k">return</span> <span class="nb">str</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">inputs</span><span class="p">)[</span><span class="mi">0</span><span class="p">])</span>
|
||||
</code></pre></div>
|
||||
<ol>
|
||||
<li><code>@use_model</code> loads and injects your model into the <code>predict_domain</code> function's <code>model</code> argument.
|
||||
You can freely reference it knowing that it is always given to the function.</li>
|
||||
</ol>
|
||||
<p>Finally, we deploy the model, inference function, and the GreatAI wrapping all of these. For that we either use: <code>great-ai deploy.ipynb</code> or build a Docker image.</p>
|
||||
|
||||
<hr>
|
||||
<div class="md-source-file">
|
||||
<small>
|
||||
|
||||
Last update:
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
|
||||
|
||||
|
||||
</small>
|
||||
|
|
@ -403,13 +721,13 @@
|
|||
|
||||
|
||||
|
||||
<a href="../how-to-guides/" class="md-footer__link md-footer__link--next" aria-label="Next: How-To Guides" rel="next">
|
||||
<a href="train/" class="md-footer__link md-footer__link--next" aria-label="Next: Train your model" rel="next">
|
||||
<div class="md-footer__title">
|
||||
<div class="md-ellipsis">
|
||||
<span class="md-footer__direction">
|
||||
Next
|
||||
</span>
|
||||
How-To Guides
|
||||
Train your model
|
||||
</div>
|
||||
</div>
|
||||
<div class="md-footer__button md-icon">
|
||||
70
tutorial/index.md
Normal file
70
tutorial/index.md
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
# Train and deploy a SOTA model
|
||||
|
||||
## Overview
|
||||
|
||||
You are going to train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the [SciBERT paper](https://arxiv.org/abs/1903.10676) in which SciBERT [achieved an F1-score of 0.6571](https://paperswithcode.com/sota/sentence-classification-on-paper-field). We are going to outperform it using a trivial text classification model: a [Linear SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html).
|
||||
|
||||
We use the same synthetic dataset derived from the [Microsoft Academic Graph](https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/). The dataset is [available here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag).
|
||||
|
||||
## Objectives
|
||||
|
||||
1. You will see how the `great_ai.utilities` can integrate into your Data Science workflow.
|
||||
2. You will use `great_ai.large_file` to version and store your trained model
|
||||
3. You will use `GreatAI` to prepare your model for a robust and responsible deployment
|
||||
|
||||
!!! success
|
||||
You are ready to start the tutorial. Feel free to come back to the [summary](#summary) section once you're finished.
|
||||
|
||||
<div style="display: flex; justify-content: space-evenly;" markdown>
|
||||
[:fontawesome-solid-chart-simple: Train it](train.ipynb){ .md-button .md-button--primary }
|
||||
|
||||
[:material-cloud-tags: Deploy it](deploy.ipynb){ .md-button .md-button--secondary }
|
||||
</div>
|
||||
|
||||
|
||||
## Summary
|
||||
|
||||
### The [training notebook](train.ipynb)
|
||||
|
||||
We load and preprocess the dataset while relying on `great_ai.utilities.clean` for the heavy-lifting. Additionally, the preprocessing is parallelised using `great_ai.utilities.simple_parallel_map`.
|
||||
|
||||
After training and evaluating a model, it is exported using `great_ai.save_model`.
|
||||
|
||||
??? tip "Using the cloud"
|
||||
To store your model remotely, you need to set your credentials before calling `save_model`.
|
||||
|
||||
For example, to use [AWS S3](https://aws.amazon.com/s3/):
|
||||
```python
|
||||
from great_ai.large_file import LargeFileS3
|
||||
|
||||
LargeFileS3.configure(
|
||||
aws_region_name='eu-west-2',
|
||||
aws_access_key_id='MY_AWS_ACCESS_KEY',
|
||||
aws_secret_access_key='MY_AWS_SECRET_KEY',
|
||||
large_files_bucket_name='my_bucket_for_models'
|
||||
)
|
||||
|
||||
from great_ai import save_model
|
||||
|
||||
save_model(model, key='my-domain-predictor')
|
||||
```
|
||||
|
||||
### The [deployment notebook](deploy.ipynb)
|
||||
|
||||
We create an inference function that can be hardened by wrapping it in a `great_ai.GreatAI` instance.
|
||||
|
||||
```python
|
||||
from great_ai import GreatAI, use_model
|
||||
from great_ai.utilities import clean
|
||||
|
||||
@GreatAI.create
|
||||
@use_model('my-domain-predictor') #(1)
|
||||
def predict_domain(sentence, model):
|
||||
inputs = [clean(sentence)]
|
||||
return str(model.predict(inputs)[0])
|
||||
```
|
||||
|
||||
1. `@use_model` loads and injects your model into the `predict_domain` function's `model` argument.
|
||||
You can freely reference it knowing that it is always given to the function.
|
||||
|
||||
Finally, we deploy the model, inference function, and the GreatAI wrapping all of these. For that we either use: `great-ai deploy.ipynb` or build a Docker image.
|
||||
2306
tutorial/train/index.html
Normal file
2306
tutorial/train/index.html
Normal file
File diff suppressed because one or more lines are too long
318
tutorial/train/train.ipynb
Normal file
318
tutorial/train/train.ipynb
Normal file
File diff suppressed because one or more lines are too long
Loading…
Add table
Add a link
Reference in a new issue