From 7305d2ead389fa9a7b5dd904cf94030595e37771 Mon Sep 17 00:00:00 2001 From: <> Date: Tue, 12 Jul 2022 17:19:00 +0000 Subject: [PATCH] Deployed b1a66cb with MkDocs version: 1.3.0 --- 404.html | 33 +- examples/simple-mag/train/index.html | 34 +- examples/simple/data/data.ipynb | 116 +- examples/simple/data/index.html | 199 +- examples/simple/deploy/deploy.ipynb | 95 +- examples/simple/deploy/index.html | 151 +- examples/simple/train/index.html | 810 +++---- examples/simple/train/train.ipynb | 1855 +++++------------ explanation/index.html | 38 +- how-to-guides/call-remote/index.html | 34 +- .../configure-service/configure-service.md | 7 +- how-to-guides/configure-service/index.html | 45 +- .../create-service/create-service.md | 8 +- how-to-guides/create-service/index.html | 54 +- how-to-guides/handle-training-data/index.html | 34 +- how-to-guides/install/index.html | 924 ++++++++ how-to-guides/install/install.md | 21 + how-to-guides/large_file/index.html | 34 +- how-to-guides/scraping/index.html | 34 +- how-to-guides/use-service/index.html | 45 +- how-to-guides/use-service/use-service.md | 5 +- index.html | 78 +- index.md | 26 +- media/og-image.png | Bin 0 -> 114474 bytes overrides/main.html | 16 + reference/index.html | 681 ++++-- reference/large-file/index.html | 34 +- reference/utilities/index.html | 193 +- reference/views/index.html | 38 +- search/search_index.json | 2 +- sitemap.xml | 5 + sitemap.xml.gz | Bin 210 -> 211 bytes tutorial/deploy/deploy.ipynb | 123 +- tutorial/deploy/index.html | 168 +- tutorial/index.html | 75 +- tutorial/index.md | 28 +- tutorial/train/index.html | 192 +- tutorial/train/train.ipynb | 151 +- 38 files changed, 3566 insertions(+), 2820 deletions(-) create mode 100644 how-to-guides/install/index.html create mode 100644 how-to-guides/install/install.md create mode 100644 media/og-image.png diff --git a/404.html b/404.html index 779024d..2bc377d 100644 --- a/404.html +++ b/404.html @@ -48,6 +48,19 @@ + + + + + + + + + @@ -346,6 +359,20 @@ +
  • + + Installation guide + +
    • + + + + + + + + +
  • How to create a GreatAI service @@ -596,7 +623,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Simple example: data engineering
    • @@ -610,7 +637,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Optimise and train a model
    • @@ -624,7 +651,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Hardening and deployment
    • diff --git a/examples/simple-mag/train/index.html b/examples/simple-mag/train/index.html index 0b8354b..a07b827 100644 --- a/examples/simple-mag/train/index.html +++ b/examples/simple-mag/train/index.html @@ -48,6 +48,19 @@ + + + + + + + + + @@ -351,6 +364,20 @@ +
  • + + Installation guide + +
    • + + + + + + + + +
  • How to create a GreatAI service @@ -601,7 +628,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Simple example: data engineering
    • @@ -615,7 +642,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Optimise and train a model
    • @@ -629,7 +656,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Hardening and deployment
    • @@ -694,6 +721,7 @@
    + diff --git a/examples/simple/data/data.ipynb b/examples/simple/data/data.ipynb index c3cc4b4..e54811b 100644 --- a/examples/simple/data/data.ipynb +++ b/examples/simple/data/data.ipynb @@ -4,17 +4,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus)\n", - "> Part 1: obtain and clean data\n", + "# Simple example: data engineering\n", "\n", - "![position of this step in the lifecycle](../diagrams/scope-data.svg)\n", - "> The blue boxes show the steps implemented in this notebook.\n", + "Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best-practices. In short, we train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus) by taking full advantage of `great-ai`. Subsequently, we create a production-ready deployment.\n", + "\n", + "![position of this step in the lifecycle](/media/scope-data.svg)\n", + "> The blue boxes show the steps of a typical AI-development lifecycle implemented in this notebook.\n", + "\n", + "Since the true scope of `great-ai` is the phase between proof-of-concept code and production-ready service, it is predominantly used in the [deployment notebook](/examples/simple/deploy). Feel free to skip there, or continue reading if you'd like to see the full picture.\n", "\n", "### Extract\n", "\n", "This can be achieved by downloading a public dataset (such as in this case), or by having a Data Engineer setup and give us access to the organisation's data.\n", "\n", - "In this case, we download the semantic scholar dataset from a public S3 bucket." + "In this example, we download the semantic scholar dataset from a public S3 bucket." ] }, { @@ -23,7 +26,7 @@ "metadata": {}, "outputs": [], "source": [ - "MAX_CHUNK_COUNT = 1" + "MAX_CHUNK_COUNT = 4" ] }, { @@ -34,7 +37,7 @@ { "data": { "text/plain": [ - "'Processing 1 out of the 6002 available chunks'" + "'Processing 4 out of the 6002 available chunks'" ] }, "execution_count": 2, @@ -69,7 +72,7 @@ "\n", "- Filter out non-English abstracts using `great_ai.utilities.predict_language`\n", "- Project it to only keep the necessary components (text and labels), clean the textual content using `great_ai.utilities.clean`\n", - "- We will speed up processing using `great_ai.utilities.parallel_map`." + "- We will speed up processing using `great_ai.utilities.simple_parallel_map`." ] }, { @@ -77,56 +80,11 @@ "execution_count": 3, "metadata": {}, "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Spacy model en_core_web_sm not found locally, downloading...\n", - "Collecting en-core-web-sm==3.3.0\n", - " Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.3.0/en_core_web_sm-3.3.0-py3-none-any.whl (12.8 MB)\n", - " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 3.6 MB/s eta 0:00:00\n", - "Requirement already satisfied: spacy<3.4.0,>=3.3.0.dev0 in ./.env/lib/python3.10/site-packages (from en-core-web-sm==3.3.0) (3.3.1)\n", - "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (4.64.0)\n", - "Requirement already satisfied: wasabi<1.1.0,>=0.9.1 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (0.9.1)\n", - "Requirement already satisfied: srsly<3.0.0,>=2.4.3 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.4.3)\n", - "Requirement already satisfied: typer<0.5.0,>=0.3.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (0.4.1)\n", - "Requirement already satisfied: setuptools in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (59.6.0)\n", - "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.0.7)\n", - "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.0.2)\n", - "Requirement already satisfied: blis<0.8.0,>=0.4.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (0.7.8)\n", - "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.0.7)\n", - "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.9 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.0.9)\n", - "Requirement already satisfied: requests<3.0.0,>=2.13.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.28.0)\n", - "Requirement already satisfied: numpy>=1.15.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.23.0)\n", - "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.0.6)\n", - "Requirement already satisfied: pathy>=0.3.5 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (0.6.1)\n", - "Requirement already satisfied: jinja2 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.1.2)\n", - "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.3.0)\n", - "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.8.2)\n", - "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.0.6)\n", - "Requirement already satisfied: thinc<8.1.0,>=8.0.14 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (8.0.17)\n", - "Requirement already satisfied: packaging>=20.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (21.3)\n", - "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.env/lib/python3.10/site-packages (from packaging>=20.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.0.9)\n", - "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in ./.env/lib/python3.10/site-packages (from pathy>=0.3.5->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (5.2.1)\n", - "Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.env/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (4.2.0)\n", - "Requirement already satisfied: charset-normalizer~=2.0.0 in ./.env/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.0.12)\n", - "Requirement already satisfied: certifi>=2017.4.17 in ./.env/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2022.6.15)\n", - "Requirement already satisfied: idna<4,>=2.5 in ./.env/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.3)\n", - "Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.env/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.26.9)\n", - "Requirement already satisfied: click<9.0.0,>=7.1.1 in ./.env/lib/python3.10/site-packages (from typer<0.5.0,>=0.3.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (8.1.3)\n", - "Requirement already satisfied: MarkupSafe>=2.0 in ./.env/lib/python3.10/site-packages (from jinja2->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.1.1)\n", - "Installing collected packages: en-core-web-sm\n", - "Successfully installed en-core-web-sm-3.3.0\n" - ] - }, { "name": "stderr", "output_type": "stream", "text": [ - "\u001b[38;5;226m2022-06-25 14:21:57,983 | WARNING | Limiting concurrency to 1 because there are only 1 chunks\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:21:57,984 | INFO | Starting parallel map (concurrency: 1, chunk size: 1)\u001b[0m\n", - "\u001b[38;5;226m2022-06-25 14:21:57,984 | WARNING | Running in series, there is no reason for parallelism\u001b[0m\n", - "100%|██████████| 1/1 [03:26<00:00, 206.86s/it]\n" + "100%|██████████| 4/4 [04:42<00:00, 70.62s/it] \n" ] } ], @@ -134,7 +92,13 @@ "from typing import List, Tuple\n", "import json\n", "import gzip\n", - "from great_ai import parallel_map, clean, is_english, predict_language\n", + "from great_ai.utilities import (\n", + " simple_parallel_map,\n", + " clean,\n", + " is_english,\n", + " predict_language,\n", + " unchunk,\n", + ")\n", "\n", "\n", "def preprocess_chunk(chunk_key: str) -> List[Tuple[str, List[str]]]:\n", @@ -161,7 +125,9 @@ " ]\n", "\n", "\n", - "preprocessed_chunks = parallel_map(preprocess_chunk, chunks)" + "preprocessed_data = unchunk(\n", + " simple_parallel_map(preprocess_chunk, chunks, concurrency=4)\n", + ")" ] }, { @@ -170,12 +136,7 @@ "metadata": {}, "outputs": [], "source": [ - "from itertools import chain\n", - "\n", - "preprocessed_data = list(chain(*preprocessed_chunks))\n", - "X, y = zip(\n", - " *preprocessed_data\n", - ") # X is the input, y is the expected (ground truth) output" + "X, y = zip(*preprocessed_data) # X is the input, y is the expected output" ] }, { @@ -194,33 +155,16 @@ "mongo_connection_string=mongodb://localhost:27017/\n", "mongo_database=my_great_ai_db\n", "```\n", - "> You can install MongoDB from [here](https://www.mongodb.com/docs/manual/installation) or [use it as a service](https://www.mongodb.com/cloud/atlas/register)\n" + "> You can install MongoDB from [here](https://www.mongodb.com/docs/manual/installation) or [use it as a service](https://www.mongodb.com/cloud/atlas/register)\n", + "\n", + "Otherwise, TinyDB is used which is just a local JSON file." ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 8, "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[38;5;226m2022-06-25 14:25:24,989 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,990 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongoDbDriver\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,991 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising LargeFileMongo\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,992 | INFO | Settings: configured ✅\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,993 | INFO | 🔩 tracing_database: MongoDbDriver\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,994 | INFO | 🔩 large_file_implementation: LargeFileMongo\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,994 | INFO | 🔩 is_production: False\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,995 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,996 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n", - "\u001b[38;5;39m2022-06-25 14:25:24,997 | INFO | 🔩 dashboard_table_size: 50\u001b[0m\n", - "\u001b[38;5;226m2022-06-25 14:25:24,998 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n", - "\u001b[38;5;226m2022-06-25 14:25:24,998 | WARNING | > Find out more at https://se-ml.github.io/practices/\u001b[0m\n" - ] - } - ], + "outputs": [], "source": [ "from great_ai import add_ground_truth\n", "\n", @@ -231,7 +175,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Next: [Part 2](train.ipynb)" + "### Next: [Part 2](/examples/simple/train)" ] } ], @@ -256,7 +200,7 @@ "orig_nbformat": 4, "vscode": { "interpreter": { - "hash": "c1f394f9662881005685eeb18d8f9f77079b1b8b9a5ece1f825bfa01fcb7f52f" + "hash": "02dd6d3afbfa9fbbe1037d64ad9014965528a1ccad21929d6e72f466389a68ad" } } }, diff --git a/examples/simple/data/index.html b/examples/simple/data/index.html index f7ca21b..5a11182 100644 --- a/examples/simple/data/index.html +++ b/examples/simple/data/index.html @@ -17,7 +17,7 @@ - Train a domain classifier on the semantic scholar dataset - GreatAI documentation + Simple example: data engineering - GreatAI documentation @@ -48,6 +48,19 @@ + + + + + + + + + @@ -70,7 +83,7 @@
    - + Skip to content @@ -103,7 +116,7 @@
    - Train a domain classifier on the semantic scholar dataset + Simple example: data engineering
    @@ -351,6 +364,20 @@ +
  • + + Installation guide + +
    • + + + + + + + + +
  • How to create a GreatAI service @@ -614,12 +641,12 @@ - Train a domain classifier on the semantic scholar dataset + Simple example: data engineering @@ -694,7 +721,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Optimise and train a model
    • @@ -708,7 +735,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Hardening and deployment
    • @@ -822,6 +849,7 @@
    + @@ -1871,13 +1899,13 @@ Licensed under the Apache License, Version 2.0.
    @@ -1903,9 +1931,9 @@ Licensed under the Apache License, Version 2.0. - 
    MAX_CHUNK_COUNT = 1
+            
    MAX_CHUNK_COUNT = 4
 
    
-
    MAX_CHUNK_COUNT = 1
    
+
    MAX_CHUNK_COUNT = 4
    @@ -1991,7 +2019,7 @@ f"Processing {len(chunks)} out of the {len(manifest.split())} available chunks"<
    -
    'Processing 1 out of the 6002 available chunks'
    +
    'Processing 4 out of the 6002 available chunks'
    @@ -2011,7 +2039,7 @@ f"Processing {len(chunks)} out of the {len(manifest.split())} available chunks"<

    Transform

    @@ -2041,7 +2069,13 @@ f"Processing {len(chunks)} out of the {len(manifest.split())} available chunks"< 
    from typing import List, Tuple
 import json
 import gzip
-from great_ai import parallel_map, clean, is_english, predict_language
+from great_ai.utilities import (
+    simple_parallel_map,
+    clean,
+    is_english,
+    predict_language,
+    unchunk,
+)
 
 
 def preprocess_chunk(chunk_key: str) -> List[Tuple[str, List[str]]]:
@@ -2068,12 +2102,20 @@ f"Processing {len(chunks)} out of the {len(manifest.split())} available chunks"<
     ]
 
 
-preprocessed_chunks = parallel_map(preprocess_chunk, chunks)
+preprocessed_data = unchunk(
+    simple_parallel_map(preprocess_chunk, chunks, concurrency=4)
+)
    from typing import List, Tuple import json import gzip -from great_ai import parallel_map, clean, is_english, predict_language +from great_ai.utilities import ( + simple_parallel_map, + clean, + is_english, + predict_language, + unchunk, +) def preprocess_chunk(chunk_key: str) -> List[Tuple[str, List[str]]]: @@ -2100,7 +2142,9 @@ def preprocess_chunk(chunk_key: str) -> List[Tuple[str, List[str]]]: ] -preprocessed_chunks = parallel_map(preprocess_chunk, chunks)
    +preprocessed_data = unchunk( + simple_parallel_map(preprocess_chunk, chunks, concurrency=4) +) @@ -2121,58 +2165,8 @@ preprocessed_chunks = parallel_map(preprocess_chunk, chunks)
    -
    -
    Spacy model en_core_web_sm not found locally, downloading...
-Collecting en-core-web-sm==3.3.0
-  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.3.0/en_core_web_sm-3.3.0-py3-none-any.whl (12.8 MB)
-     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 3.6 MB/s eta 0:00:00
-Requirement already satisfied: spacy<3.4.0,>=3.3.0.dev0 in ./.env/lib/python3.10/site-packages (from en-core-web-sm==3.3.0) (3.3.1)
-Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (4.64.0)
-Requirement already satisfied: wasabi<1.1.0,>=0.9.1 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (0.9.1)
-Requirement already satisfied: srsly<3.0.0,>=2.4.3 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.4.3)
-Requirement already satisfied: typer<0.5.0,>=0.3.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (0.4.1)
-Requirement already satisfied: setuptools in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (59.6.0)
-Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.0.7)
-Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.0.2)
-Requirement already satisfied: blis<0.8.0,>=0.4.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (0.7.8)
-Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.0.7)
-Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.9 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.0.9)
-Requirement already satisfied: requests<3.0.0,>=2.13.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.28.0)
-Requirement already satisfied: numpy>=1.15.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.23.0)
-Requirement already satisfied: cymem<2.1.0,>=2.0.2 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.0.6)
-Requirement already satisfied: pathy>=0.3.5 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (0.6.1)
-Requirement already satisfied: jinja2 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.1.2)
-Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.3.0)
-Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.8.2)
-Requirement already satisfied: preshed<3.1.0,>=3.0.2 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.0.6)
-Requirement already satisfied: thinc<8.1.0,>=8.0.14 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (8.0.17)
-Requirement already satisfied: packaging>=20.0 in ./.env/lib/python3.10/site-packages (from spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (21.3)
-Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.env/lib/python3.10/site-packages (from packaging>=20.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.0.9)
-Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in ./.env/lib/python3.10/site-packages (from pathy>=0.3.5->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (5.2.1)
-Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.env/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (4.2.0)
-Requirement already satisfied: charset-normalizer~=2.0.0 in ./.env/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.0.12)
-Requirement already satisfied: certifi>=2017.4.17 in ./.env/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2022.6.15)
-Requirement already satisfied: idna<4,>=2.5 in ./.env/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (3.3)
-Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.env/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (1.26.9)
-Requirement already satisfied: click<9.0.0,>=7.1.1 in ./.env/lib/python3.10/site-packages (from typer<0.5.0,>=0.3.0->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (8.1.3)
-Requirement already satisfied: MarkupSafe>=2.0 in ./.env/lib/python3.10/site-packages (from jinja2->spacy<3.4.0,>=3.3.0.dev0->en-core-web-sm==3.3.0) (2.1.1)
-Installing collected packages: en-core-web-sm
-Successfully installed en-core-web-sm-3.3.0
-
    -
    - - -
    - - - - -
    @@ -2203,19 +2197,9 @@ Successfully installed en-core-web-sm-3.3.0 - 
    from itertools import chain
-
-preprocessed_data = list(chain(*preprocessed_chunks))
-X, y = zip(
-    *preprocessed_data
-)  # X is the input, y is the expected (ground truth) output
+            
    X, y = zip(*preprocessed_data)  # X is the input, y is the expected output
 
    
-
    from itertools import chain
-
-preprocessed_data = list(chain(*preprocessed_chunks))
-X, y = zip(
-    *preprocessed_data
-)  # X is the input, y is the expected (ground truth) output
    
+
    X, y = zip(*preprocessed_data)  # X is the input, y is the expected output
    @@ -2238,17 +2222,18 @@ X, y = zip(

    You can install MongoDB from here or use it as a service

    +

    Otherwise, TinyDB is used which is just a local JSON file.

    -
    - @@ -2333,7 +2284,7 @@ add_ground_truth(X, y, train_split_ratio=0.8, test_split_ratio=0.2)
    Last update: - July 11, 2022 + July 12, 2022 @@ -2370,13 +2321,13 @@ add_ground_truth(X, y, train_split_ratio=0.8, test_split_ratio=0.2) -
    + Skip to content @@ -103,7 +116,7 @@
    - Train a domain classifier on the semantic scholar dataset + Hardening and deployment
    @@ -351,6 +364,20 @@ +
  • + + Installation guide + +
    • + + + + + + + + +
  • How to create a GreatAI service @@ -605,7 +632,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Simple example: data engineering
    • @@ -619,7 +646,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Optimise and train a model
    • @@ -642,12 +669,12 @@ - Train a domain classifier on the semantic scholar dataset + Hardening and deployment @@ -665,6 +692,13 @@
    @@ -1994,20 +2037,22 @@ def get_label(
    @@ -2024,7 +2069,8 @@ def get_label(
    @@ -2137,8 +2183,7 @@ def get_label(
    -
    2022-06-25 14:57:21,004 |     INFO | Starting parallel map (concurrency: 12, chunk size: 26)
-100%|██████████| 3054/3054 [00:31<00:00, 96.38it/s] 
+
    100%|██████████| 12332/12332 [01:27<00:00, 141.08it/s]
    @@ -2152,29 +2197,29 @@ def get_label( 
                           precision    recall  f1-score   support
 
-                  Art       0.35      0.27      0.31        33
-              Biology       0.77      0.84      0.80       299
-             Business       0.44      0.61      0.51        82
-            Chemistry       0.79      0.67      0.72       289
-     Computer Science       0.74      0.79      0.77       341
-            Economics       0.65      0.56      0.60        61
-          Engineering       0.53      0.52      0.53       202
-Environmental Science       0.51      0.49      0.50        61
-            Geography       0.54      0.26      0.35        58
-              Geology       0.69      0.56      0.62        48
-              History       0.36      0.17      0.23        29
-    Materials Science       0.68      0.83      0.74       248
-          Mathematics       0.77      0.57      0.66       122
-             Medicine       0.97      0.79      0.87       715
-           Philosophy       0.50      0.05      0.10        19
-              Physics       0.72      0.75      0.74       162
-    Political Science       0.39      0.49      0.43        57
-           Psychology       0.49      0.81      0.61       141
-            Sociology       0.32      0.53      0.40        87
+                  Art       0.43      0.35      0.39       125
+              Biology       0.77      0.83      0.80      1209
+             Business       0.50      0.71      0.59       312
+            Chemistry       0.80      0.67      0.73      1194
+     Computer Science       0.77      0.76      0.77      1293
+            Economics       0.64      0.58      0.61       251
+          Engineering       0.55      0.53      0.54       815
+Environmental Science       0.54      0.56      0.55       230
+            Geography       0.54      0.44      0.48       277
+              Geology       0.76      0.67      0.71       228
+              History       0.29      0.21      0.24       102
+    Materials Science       0.73      0.81      0.77      1053
+          Mathematics       0.80      0.70      0.75       551
+             Medicine       0.95      0.77      0.85      2794
+           Philosophy       0.61      0.12      0.21        88
+              Physics       0.67      0.76      0.72       605
+    Political Science       0.43      0.52      0.47       297
+           Psychology       0.53      0.80      0.63       600
+            Sociology       0.32      0.60      0.42       308
 
-             accuracy                           0.70      3054
-            macro avg       0.59      0.56      0.55      3054
-         weighted avg       0.72      0.70      0.70      3054
+             accuracy                           0.71     12332
+            macro avg       0.61      0.60      0.59     12332
+         weighted avg       0.73      0.71      0.71     12332
    @@ -2189,7 +2234,7 @@ Environmental Science 0.51 0.49 0.50 61
    - @@ -2212,7 +2257,7 @@ class=" Last update: - July 11, 2022 + July 12, 2022 @@ -2233,7 +2278,7 @@ class="
    diff --git a/examples/simple/train/index.html b/examples/simple/train/index.html index 60a39d4..815fde6 100644 --- a/examples/simple/train/index.html +++ b/examples/simple/train/index.html @@ -17,7 +17,7 @@ - Train a domain classifier on the semantic scholar dataset - GreatAI documentation + Optimise and train a model - GreatAI documentation @@ -48,6 +48,19 @@ + + + + + + + + + @@ -70,7 +83,7 @@
    - + Skip to content @@ -103,7 +116,7 @@
    - Train a domain classifier on the semantic scholar dataset + Optimise and train a model
    @@ -351,6 +364,20 @@ +
  • + + Installation guide + +
    • + + + + + + + + +
  • How to create a GreatAI service @@ -605,7 +632,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Simple example: data engineering
    • @@ -628,12 +655,12 @@ - Train a domain classifier on the semantic scholar dataset + Optimise and train a model @@ -670,19 +697,13 @@ Export the model using GreatAI - - @@ -701,7 +722,7 @@
  • - Train a domain classifier on the semantic scholar dataset + Hardening and deployment
    • @@ -781,19 +802,13 @@ Export the model using GreatAI - - @@ -808,6 +823,7 @@
    + @@ -1857,25 +1873,12 @@ Licensed under the Apache License, Version 2.0.
    -
    - - -
    - @@ -1932,18 +1935,19 @@ y = [domain for d in data for domain in d.feedback]
    -
    2022-06-25 14:50:29,879 |  WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️
-2022-06-25 14:50:29,880 |     INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongoDbDriver
-2022-06-25 14:50:29,881 |     INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising LargeFileMongo
-2022-06-25 14:50:29,881 |     INFO | Settings: configured ✅
-2022-06-25 14:50:29,882 |     INFO | 🔩 tracing_database: MongoDbDriver
-2022-06-25 14:50:29,883 |     INFO | 🔩 large_file_implementation: LargeFileMongo
-2022-06-25 14:50:29,883 |     INFO | 🔩 is_production: False
-2022-06-25 14:50:29,884 |     INFO | 🔩 should_log_exception_stack: True
-2022-06-25 14:50:29,884 |     INFO | 🔩 prediction_cache_size: 512
-2022-06-25 14:50:29,885 |     INFO | 🔩 dashboard_table_size: 50
-2022-06-25 14:50:29,885 |  WARNING | You still need to check whether you follow all best practices before trusting your deployment.
-2022-06-25 14:50:29,885 |  WARNING | > Find out more at https://se-ml.github.io/practices/
+
    Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️
+Cannot find credentials files, defaulting to using ParallelTinyDbDriver
+The selected tracing database (ParallelTinyDbDriver) is not recommended for production
+Cannot find credentials files, defaulting to using LargeFileLocal
+GreatAI (v0.1.4): configured ✅
+  🔩 tracing_database: ParallelTinyDbDriver
+  🔩 large_file_implementation: LargeFileLocal
+  🔩 is_production: False
+  🔩 should_log_exception_stack: True
+  🔩 prediction_cache_size: 512
+  🔩 dashboard_table_size: 50
+You still need to check whether you follow all best practices before trusting your deployment.
+> Find out more at https://se-ml.github.io/practices
    @@ -1959,7 +1963,7 @@ y = [domain for d in data for domain in d.feedback]
    - - 
    import pandas as pd
-from collections import Counter
-import plotly.express as px
+            
    from collections import Counter
+import matplotlib.pyplot as plt
 
-df = pd.DataFrame(Counter(y).most_common(), columns=["domain", "count"])
-px.bar(df, "domain", "count", width=1200, height=400).show()
+domains, counts = zip(*Counter(y).most_common())
+
+# Configure matplotlib to have nice, high-resolution charts
+%matplotlib inline
+plt.rcParams["figure.figsize"] = (20, 5)
+plt.rcParams["figure.facecolor"] = "white"
+plt.rcParams["font.size"] = 12
+
+plt.xticks(rotation=90)
+plt.bar(domains, counts)
+None
 
    
-
    import pandas as pd
-from collections import Counter
-import plotly.express as px
+
    from collections import Counter
+import matplotlib.pyplot as plt
 
-df = pd.DataFrame(Counter(y).most_common(), columns=["domain", "count"])
-px.bar(df, "domain", "count", width=1200, height=400).show()
    
+domains, counts = zip(*Counter(y).most_common())
+
+# Configure matplotlib to have nice, high-resolution charts
+%matplotlib inline
+plt.rcParams["figure.figsize"] = (20, 5)
+plt.rcParams["figure.facecolor"] = "white"
+plt.rcParams["font.size"] = 12
+
+plt.xticks(rotation=90)
+plt.bar(domains, counts)
+None
    @@ -2009,6 +2029,12 @@ px.bar(df, "domain", "count", width=1200, height=400).show() +
    + +
    @@ -2214,434 +2240,434 @@ results.sort_values("rank_test_score") 7 - 1.962260 - 0.147449 - 0.935357 - 0.063659 + 9.447799 + 0.785535 + 4.260490 + 0.180851 0.5 False 0.05 20 {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.485030 - 0.463849 - 0.481840 - 0.476906 - 0.009324 + 0.517618 + 0.518688 + 0.513803 + 0.516703 + 0.002096 1 10 - 1.942605 - 0.111027 - 0.952361 - 0.066812 + 9.750011 + 0.671769 + 4.791153 + 0.544862 0.5 False 0.1 20 {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.482890 - 0.459556 - 0.479362 - 0.473936 - 0.010270 + 0.513254 + 0.517940 + 0.509116 + 0.513437 + 0.003605 2 - 19 - 2.145152 - 0.068978 - 1.002291 - 0.047358 - 1 + 11 + 9.398448 + 0.656403 + 5.194527 + 0.303382 + 0.5 False - 0.05 - 20 - {'classifier__alpha': 1, 'classifier__fit_prio... - 0.467330 - 0.442994 - 0.464302 - 0.458208 - 0.010829 + 0.1 + 100 + {'classifier__alpha': 0.5, 'classifier__fit_pr... + 0.508306 + 0.502244 + 0.501839 + 0.504129 + 0.002958 3 - 22 - 1.971888 - 0.126950 - 0.739795 - 0.071551 - 1 + 8 + 9.285461 + 0.869226 + 5.030563 + 0.305472 + 0.5 False - 0.1 - 20 - {'classifier__alpha': 1, 'classifier__fit_prio... - 0.454830 - 0.422902 - 0.450677 - 0.442803 - 0.014174 + 0.05 + 100 + {'classifier__alpha': 0.5, 'classifier__fit_pr... + 0.504070 + 0.502116 + 0.500603 + 0.502263 + 0.001419 4 - 6 - 1.861275 - 0.013389 - 1.058907 - 0.111122 - 0.5 + 19 + 9.662259 + 0.234167 + 4.854762 + 0.126622 + 1 False 0.05 + 20 + {'classifier__alpha': 1, 'classifier__fit_prio... + 0.492205 + 0.504466 + 0.495489 + 0.497387 + 0.005182 5 - {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.456127 - 0.422456 - 0.443827 - 0.440803 - 0.013912 - 5 - - - 11 - 1.825397 - 0.105754 - 0.892227 - 0.057003 - 0.5 - False - 0.1 - 100 - {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.438232 - 0.440464 - 0.422667 - 0.433788 - 0.007916 - 6 23 - 1.693333 - 0.009667 - 0.501491 - 0.006545 + 9.024540 + 0.469833 + 3.235823 + 0.173482 1 False 0.1 100 {'classifier__alpha': 1, 'classifier__fit_prio... - 0.433915 - 0.439470 - 0.416031 - 0.429805 - 0.010001 - 7 - - - 8 - 2.008045 - 0.145330 - 0.944559 - 0.155925 - 0.5 - False - 0.05 - 100 - {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.436178 - 0.425724 - 0.418396 - 0.426766 - 0.007297 - 8 + 0.501203 + 0.493465 + 0.492361 + 0.495676 + 0.003934 + 6 20 - 1.749200 - 0.022959 - 0.889532 - 0.047517 + 9.304917 + 0.192977 + 4.370798 + 0.255480 1 False 0.05 100 {'classifier__alpha': 1, 'classifier__fit_prio... - 0.428215 - 0.425398 - 0.411051 - 0.421555 - 0.007516 + 0.498869 + 0.494357 + 0.492431 + 0.495219 + 0.002698 + 7 + + + 22 + 9.642570 + 0.438559 + 3.332855 + 0.435930 + 1 + False + 0.1 + 20 + {'classifier__alpha': 1, 'classifier__fit_prio... + 0.479585 + 0.496964 + 0.485209 + 0.487253 + 0.007241 + 8 + + + 6 + 10.047471 + 0.473215 + 4.358744 + 0.051892 + 0.5 + False + 0.05 + 5 + {'classifier__alpha': 0.5, 'classifier__fit_pr... + 0.472578 + 0.486362 + 0.481865 + 0.480268 + 0.005739 9 - 9 - 1.960889 - 0.098004 - 0.985957 - 0.080925 + 5 + 10.287111 + 0.349778 + 4.908435 + 0.099558 0.5 - False + True 0.1 - 5 + 100 {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.430638 - 0.406619 - 0.420213 - 0.419157 - 0.009834 + 0.474821 + 0.474690 + 0.478096 + 0.475869 + 0.001576 10 - 18 - 1.807799 - 0.064891 - 0.881872 - 0.030810 - 1 - False + 2 + 10.997282 + 0.851917 + 6.131619 + 0.274400 + 0.5 + True 0.05 - 5 - {'classifier__alpha': 1, 'classifier__fit_prio... - 0.402402 - 0.372353 - 0.386189 - 0.386981 - 0.012280 + 100 + {'classifier__alpha': 0.5, 'classifier__fit_pr... + 0.476218 + 0.471377 + 0.478548 + 0.475381 + 0.002987 11 + + 9 + 9.147621 + 0.290876 + 5.392678 + 0.390403 + 0.5 + False + 0.1 + 5 + {'classifier__alpha': 0.5, 'classifier__fit_pr... + 0.458994 + 0.475008 + 0.469998 + 0.468000 + 0.006689 + 12 + 1 - 2.009232 - 0.043125 - 0.899676 - 0.036977 + 10.299674 + 0.486001 + 5.776907 + 0.462777 0.5 True 0.05 20 {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.389797 - 0.372619 - 0.388358 - 0.383591 - 0.007781 - 12 + 0.461036 + 0.466296 + 0.467280 + 0.464871 + 0.002741 + 13 4 - 1.868087 - 0.094739 - 1.005353 - 0.101466 + 10.771393 + 0.330218 + 4.961415 + 0.454726 0.5 True 0.1 20 {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.379323 - 0.364667 - 0.379060 - 0.374350 - 0.006848 - 13 + 0.454689 + 0.463340 + 0.457743 + 0.458591 + 0.003582 + 14 + + + 14 + 9.969123 + 0.715883 + 4.382175 + 0.216170 + 1 + True + 0.05 + 100 + {'classifier__alpha': 1, 'classifier__fit_prio... + 0.444039 + 0.449822 + 0.450524 + 0.448128 + 0.002906 + 15 + + + 17 + 9.439775 + 1.333323 + 4.914944 + 0.292653 + 1 + True + 0.1 + 100 + {'classifier__alpha': 1, 'classifier__fit_prio... + 0.440663 + 0.448419 + 0.443441 + 0.444174 + 0.003209 + 16 + + + 18 + 10.025776 + 0.198347 + 5.375965 + 0.220059 + 1 + False + 0.05 + 5 + {'classifier__alpha': 1, 'classifier__fit_prio... + 0.418284 + 0.428063 + 0.428696 + 0.425014 + 0.004766 + 17 + + + 13 + 9.251913 + 0.441015 + 5.085037 + 0.224299 + 1 + True + 0.05 + 20 + {'classifier__alpha': 1, 'classifier__fit_prio... + 0.400471 + 0.406275 + 0.406521 + 0.404422 + 0.002796 + 18 21 - 1.958430 - 0.039639 - 0.890963 - 0.012546 + 9.173751 + 0.112242 + 4.271786 + 0.293579 1 False 0.1 5 {'classifier__alpha': 1, 'classifier__fit_prio... - 0.366936 - 0.343361 - 0.363883 - 0.358060 - 0.010468 - 14 - - - 5 - 1.940692 - 0.018320 - 0.898865 - 0.030651 - 0.5 - True - 0.1 - 100 - {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.354850 - 0.349237 - 0.342737 - 0.348941 - 0.004950 - 15 - - - 2 - 1.855691 - 0.029506 - 0.866492 - 0.038048 - 0.5 - True - 0.05 - 100 - {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.349005 - 0.341832 - 0.328617 - 0.339818 - 0.008444 - 16 - - - 17 - 1.798559 - 0.103497 - 0.888273 - 0.069050 - 1 - True - 0.1 - 100 - {'classifier__alpha': 1, 'classifier__fit_prio... - 0.312332 - 0.297655 - 0.307471 - 0.305819 - 0.006104 - 17 - - - 14 - 2.016041 - 0.232138 - 0.967630 - 0.146144 - 1 - True - 0.05 - 100 - {'classifier__alpha': 1, 'classifier__fit_prio... - 0.304942 - 0.296921 - 0.297121 - 0.299661 - 0.003735 - 18 - - - 13 - 1.829513 - 0.112645 - 0.885848 - 0.027726 - 1 - True - 0.05 - 20 - {'classifier__alpha': 1, 'classifier__fit_prio... - 0.301539 - 0.285396 - 0.297272 - 0.294736 - 0.006830 + 0.393559 + 0.402869 + 0.406431 + 0.400953 + 0.005427 19 + + 16 + 9.683013 + 0.600089 + 4.924559 + 0.321389 + 1 + True + 0.1 + 20 + {'classifier__alpha': 1, 'classifier__fit_prio... + 0.387535 + 0.392661 + 0.393992 + 0.391396 + 0.002784 + 20 + 0 - 1.905362 - 0.018052 - 0.885552 - 0.023985 + 9.835260 + 0.220524 + 6.034349 + 0.586497 0.5 True 0.05 5 {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.295635 - 0.276759 - 0.296270 - 0.289555 - 0.009052 - 20 - - - 16 - 1.793688 - 0.049995 - 0.921301 - 0.060980 - 1 - True - 0.1 - 20 - {'classifier__alpha': 1, 'classifier__fit_prio... - 0.286746 - 0.272260 - 0.277084 - 0.278696 - 0.006023 + 0.379471 + 0.381348 + 0.386078 + 0.382299 + 0.002780 21 3 - 2.078568 - 0.045549 - 0.963691 - 0.048281 + 11.905621 + 0.329256 + 5.571229 + 0.541937 0.5 True 0.1 5 {'classifier__alpha': 0.5, 'classifier__fit_pr... - 0.276452 - 0.265509 - 0.268949 - 0.270303 - 0.004569 + 0.368234 + 0.367599 + 0.372849 + 0.369561 + 0.002339 22 12 - 1.839506 - 0.048910 - 0.921812 - 0.010374 + 10.378068 + 0.227200 + 5.247946 + 0.126687 1 True 0.05 5 {'classifier__alpha': 1, 'classifier__fit_prio... - 0.183196 - 0.186144 - 0.180323 - 0.183221 - 0.002376 + 0.278087 + 0.278157 + 0.278143 + 0.278129 + 0.000030 23 15 - 1.909279 - 0.110639 - 1.056087 - 0.129738 + 10.375661 + 0.139826 + 5.239350 + 0.298795 1 True 0.1 5 {'classifier__alpha': 1, 'classifier__fit_prio... - 0.165277 - 0.165840 - 0.167088 - 0.166068 - 0.000757 + 0.259347 + 0.262720 + 0.258716 + 0.260261 + 0.001758 24 @@ -2790,18 +2816,10 @@ save_model(classifier, key="small-domain-prediction", keep_last_n=5)
    -
    2022-06-25 14:50:53,592 |     INFO | Copying file for small-domain-prediction-0
-2022-06-25 14:50:53,613 |     INFO | Compressing small-domain-prediction-0
-2022-06-25 14:50:53,917 |     INFO | Uploading /tmp/tmpvxez8op8/small-domain-prediction-0.tar.gz to Mongo (GridFS)
-2022-06-25 14:50:53,972 |     INFO | Uploading small-domain-prediction-0.tar.gz 0.26/1.85 MB (14.1%)
-2022-06-25 14:50:53,974 |     INFO | Uploading small-domain-prediction-0.tar.gz 0.52/1.85 MB (28.2%)
-2022-06-25 14:50:53,975 |     INFO | Uploading small-domain-prediction-0.tar.gz 0.78/1.85 MB (42.3%)
-2022-06-25 14:50:53,977 |     INFO | Uploading small-domain-prediction-0.tar.gz 1.04/1.85 MB (56.4%)
-2022-06-25 14:50:53,979 |     INFO | Uploading small-domain-prediction-0.tar.gz 1.31/1.85 MB (70.5%)
-2022-06-25 14:50:53,980 |     INFO | Uploading small-domain-prediction-0.tar.gz 1.57/1.85 MB (84.7%)
-2022-06-25 14:50:53,982 |     INFO | Uploading small-domain-prediction-0.tar.gz 1.83/1.85 MB (98.8%)
-2022-06-25 14:50:53,982 |     INFO | Uploading small-domain-prediction-0.tar.gz 1.85/1.85 MB (100.0%)
-2022-06-25 14:50:53,985 |     INFO | Model small-domain-prediction uploaded with version 0
+
    Fetching cached versions of small-domain-prediction
+Copying file for small-domain-prediction-1
+Compressing small-domain-prediction-1
+Model small-domain-prediction uploaded with version 1
    @@ -2815,7 +2833,7 @@ save_model(classifier, key="small-domain-prediction", keep_last_n=5)
    -
    'small-domain-prediction:0'
    +
    'small-domain-prediction:1'
    @@ -2832,7 +2850,7 @@ save_model(classifier, key="small-domain-prediction", keep_last_n=5)
    @@ -2846,7 +2864,7 @@ save_model(classifier, key="small-domain-prediction", keep_last_n=5) Last update: - July 11, 2022 + July 12, 2022 @@ -2867,7 +2885,7 @@ save_model(classifier, key="small-domain-prediction", keep_last_n=5)