andras/great-ai
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 4 Packages Wiki Activity Actions

Deployed 7a71510 with MkDocs version: 1.3.0

Browse source
This commit is contained in:
2022-07-13 11:09:23 +00:00
parent f673e05f40
commit 4d4c32baf6
33 changed files with 641 additions and 2682 deletions
Download patch file Download diff file Expand all files Collapse all files

14
404.html
View file

@ -452,20 +452,6 @@
<li class="md-nav__item">
<a href="/how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

52
examples/scibert/analyse.ipynb
View file

File diff suppressed because one or more lines are too long

14
examples/scibert/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
examples/scibert/views/__init__/index.html
View file

@ -452,20 +452,6 @@
<li class="md-nav__item">
<a href="../../../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
examples/scibert/views/evaluated_sentence/index.html
View file

@ -452,20 +452,6 @@
<li class="md-nav__item">
<a href="../../../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
examples/scibert/views/match/index.html
View file

@ -452,20 +452,6 @@
<li class="md-nav__item">
<a href="../../../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
examples/simple-mag/train/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
examples/simple/data/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
examples/simple/deploy/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
examples/simple/train/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
explanation/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

18
how-to-guides/call-remote/index.html
View file

@ -540,20 +540,6 @@
<li class="md-nav__item">
<a href="../scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>
@ -1003,13 +989,13 @@
<a href="../scraping/" class="md-footer__link md-footer__link--next" aria-label="Next: Scraping" rel="next"> <a href="../../reference/" class="md-footer__link md-footer__link--next" aria-label="Next: GreatAI reference" rel="next">
<div class="md-footer__title"> <div class="md-footer__title">
<div class="md-ellipsis"> <div class="md-ellipsis">
<span class="md-footer__direction"> <span class="md-footer__direction">
Next Next
</span> </span>
Scraping GreatAI reference
</div> </div>
</div> </div>
<div class="md-footer__button md-icon"> <div class="md-footer__button md-icon">

14
how-to-guides/configure-service/index.html
View file

@ -547,20 +547,6 @@
<li class="md-nav__item">
<a href="../scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
how-to-guides/create-service/index.html
View file

@ -541,20 +541,6 @@
<li class="md-nav__item">
<a href="../scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
how-to-guides/handle-training-data/index.html
View file

@ -514,20 +514,6 @@
<li class="md-nav__item">
<a href="../scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
how-to-guides/install/index.html
View file

@ -500,20 +500,6 @@
<li class="md-nav__item">
<a href="../scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
how-to-guides/large_file/index.html
View file

@ -574,20 +574,6 @@
<li class="md-nav__item">
<a href="../scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

2058
how-to-guides/scraping/index.html
View file

File diff suppressed because one or more lines are too long

104
how-to-guides/scraping/scraping.ipynb
View file

@ -1,104 +0,0 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import httpx\n",
"\n",
"transport = httpx.AsyncHTTPTransport(retries=2)\n",
"limits = httpx.Limits(max_connections=None)\n",
"timeout = httpx.Timeout(connect=60.0, read=300, write=60, pool=None)\n",
"async with httpx.AsyncClient(\n",
" transport=transport, limits=limits, timeout=timeout\n",
") as client:\n",
" pass"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver\u001b[0m\n",
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production\u001b[0m\n",
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal\u001b[0m\n",
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | GreatAI (v0.1.3): configured ✅\u001b[0m\n",
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 tracing_database: ParallelTinyDbDriver\u001b[0m\n",
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 large_file_implementation: LargeFileLocal\u001b[0m\n",
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 is_production: False\u001b[0m\n",
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
"\u001b[38;5;39m2022-07-11 19:00:27 | INFO | 🔩 dashboard_table_size: 50\u001b[0m\n",
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
"\u001b[38;5;226m2022-07-11 19:00:27 | WARNING | > Find out more at https://se-ml.github.io/practices\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"Trace[str]({'created': '2022-07-11T17:00:27.064568',\n",
" 'exception': None,\n",
" 'feedback': None,\n",
" 'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},\n",
" 'models': [],\n",
" 'original_execution_time_ms': 0.0896,\n",
" 'output': 'Hi Bob',\n",
" 'tags': ['greeter', 'online', 'development'],\n",
" 'trace_id': '8ff5b268-2613-4e85-96ae-f248666a051f'})"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from great_ai import configure, RouteConfig\n",
"from great_ai import GreatAI\n",
"\n",
"\n",
"@GreatAI.create\n",
"def greeter(your_name: str) -> str:\n",
" return f\"Hi {your_name}\"\n",
"\n",
"\n",
"greeter(\"Bob\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.10.4 ('.env': venv)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
},
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "02dd6d3afbfa9fbbe1037d64ad9014965528a1ccad21929d6e72f466389a68ad"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}

14
how-to-guides/use-service/index.html
View file

@ -561,20 +561,6 @@
<li class="md-nav__item">
<a href="../scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
index.html
View file

@ -529,20 +529,6 @@
<li class="md-nav__item">
<a href="how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

BIN
objects.inv
View file

Binary file not shown.

27
reference/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>
@ -1634,7 +1620,7 @@ registered through use_model.</p></td>
<a id="__codelineno-0-190" name="__codelineno-0-190"></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">List</span><span class="p">[</span><span class="n">Trace</span><span class="p">[</span><span class="n">V</span><span class="p">]]:</span> <a id="__codelineno-0-190" name="__codelineno-0-190"></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">List</span><span class="p">[</span><span class="n">Trace</span><span class="p">[</span><span class="n">V</span><span class="p">]]:</span>
<a id="__codelineno-0-191" name="__codelineno-0-191"></a> <span class="sd">&quot;&quot;&quot;Map the wrapped function over a list of input_values (`batch`).</span> <a id="__codelineno-0-191" name="__codelineno-0-191"></a> <span class="sd">&quot;&quot;&quot;Map the wrapped function over a list of input_values (`batch`).</span>
<a id="__codelineno-0-192" name="__codelineno-0-192"></a> <a id="__codelineno-0-192" name="__codelineno-0-192"></a>
<a id="__codelineno-0-193" name="__codelineno-0-193"></a><span class="sd"> A wrapper over [parallel_map][great_ai.utilities.parallel_map]</span> <a id="__codelineno-0-193" name="__codelineno-0-193"></a><span class="sd"> A wrapper over [parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map]</span>
<a id="__codelineno-0-194" name="__codelineno-0-194"></a><span class="sd"> providing type-safety and a progressbar through tqdm.</span> <a id="__codelineno-0-194" name="__codelineno-0-194"></a><span class="sd"> providing type-safety and a progressbar through tqdm.</span>
<a id="__codelineno-0-195" name="__codelineno-0-195"></a> <a id="__codelineno-0-195" name="__codelineno-0-195"></a>
<a id="__codelineno-0-196" name="__codelineno-0-196"></a><span class="sd"> Args:</span> <a id="__codelineno-0-196" name="__codelineno-0-196"></a><span class="sd"> Args:</span>
@ -1971,7 +1957,7 @@ property.</p>
<div class="doc doc-contents "> <div class="doc doc-contents ">
<p>Map the wrapped function over a list of input_values (<code>batch</code>).</p> <p>Map the wrapped function over a list of input_values (<code>batch</code>).</p>
<p>A wrapper over [parallel_map][great_ai.utilities.parallel_map] <p>A wrapper over <a class="autorefs autorefs-internal" href="utilities/#great_ai.utilities.parallel_map.parallel_map.parallel_map">parallel_map</a>
providing type-safety and a progressbar through tqdm.</p> providing type-safety and a progressbar through tqdm.</p>
<p><strong>Parameters:</strong></p> <p><strong>Parameters:</strong></p>
@ -2106,7 +2092,7 @@ evaluations run part of the CI.</p></td>
<a id="__codelineno-0-190" name="__codelineno-0-190"></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">List</span><span class="p">[</span><span class="n">Trace</span><span class="p">[</span><span class="n">V</span><span class="p">]]:</span> <a id="__codelineno-0-190" name="__codelineno-0-190"></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">List</span><span class="p">[</span><span class="n">Trace</span><span class="p">[</span><span class="n">V</span><span class="p">]]:</span>
<a id="__codelineno-0-191" name="__codelineno-0-191"></a> <span class="sd">&quot;&quot;&quot;Map the wrapped function over a list of input_values (`batch`).</span> <a id="__codelineno-0-191" name="__codelineno-0-191"></a> <span class="sd">&quot;&quot;&quot;Map the wrapped function over a list of input_values (`batch`).</span>
<a id="__codelineno-0-192" name="__codelineno-0-192"></a> <a id="__codelineno-0-192" name="__codelineno-0-192"></a>
<a id="__codelineno-0-193" name="__codelineno-0-193"></a><span class="sd"> A wrapper over [parallel_map][great_ai.utilities.parallel_map]</span> <a id="__codelineno-0-193" name="__codelineno-0-193"></a><span class="sd"> A wrapper over [parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map]</span>
<a id="__codelineno-0-194" name="__codelineno-0-194"></a><span class="sd"> providing type-safety and a progressbar through tqdm.</span> <a id="__codelineno-0-194" name="__codelineno-0-194"></a><span class="sd"> providing type-safety and a progressbar through tqdm.</span>
<a id="__codelineno-0-195" name="__codelineno-0-195"></a> <a id="__codelineno-0-195" name="__codelineno-0-195"></a>
<a id="__codelineno-0-196" name="__codelineno-0-196"></a><span class="sd"> Args:</span> <a id="__codelineno-0-196" name="__codelineno-0-196"></a><span class="sd"> Args:</span>
@ -3535,7 +3521,8 @@ scikit-learn)</p></td>
<td> <td>
<code>float</code> <code>float</code>
</td> </td>
<td><p>The probability-weight of giving each trace the</p></td> <td><p>The probability-weight of giving each trace the
<code>validation</code> tag.</p></td>
<td> <td>
<code>0</code> <code>0</code>
</td> </td>
@ -4958,7 +4945,7 @@ not advised for production use.</p>
<nav class="md-footer__inner md-grid" aria-label="Footer" > <nav class="md-footer__inner md-grid" aria-label="Footer" >
<a href="../how-to-guides/scraping/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Scraping" rel="prev"> <a href="../how-to-guides/call-remote/" class="md-footer__link md-footer__link--prev" aria-label="Previous: How to call remote GreatAI instances" rel="prev">
<div class="md-footer__button md-icon"> <div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</div> </div>
@ -4967,7 +4954,7 @@ not advised for production use.</p>
<span class="md-footer__direction"> <span class="md-footer__direction">
Previous Previous
</span> </span>
Scraping How to call remote GreatAI instances
</div> </div>
</div> </div>
</a> </a>

14
reference/large-file/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

735
reference/utilities/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>
@ -580,6 +566,34 @@
get_sentences() get_sentences()
</a> </a>
</li>
<li class="md-nav__item">
<a href="#great_ai.utilities.language.predict_language.predict_language" class="md-nav__link">
predict_language()
</a>
</li>
<li class="md-nav__item">
<a href="#great_ai.utilities.language.english_name_of_language.english_name_of_language" class="md-nav__link">
english_name_of_language()
</a>
</li>
<li class="md-nav__item">
<a href="#great_ai.utilities.language.is_english.is_english" class="md-nav__link">
is_english()
</a>
</li>
<li class="md-nav__item">
<a href="#great_ai.utilities.evaluate_ranking.evaluate_ranking.evaluate_ranking" class="md-nav__link">
evaluate_ranking()
</a>
</li> </li>
</ul> </ul>
@ -922,6 +936,34 @@
get_sentences() get_sentences()
</a> </a>
</li>
<li class="md-nav__item">
<a href="#great_ai.utilities.language.predict_language.predict_language" class="md-nav__link">
predict_language()
</a>
</li>
<li class="md-nav__item">
<a href="#great_ai.utilities.language.english_name_of_language.english_name_of_language" class="md-nav__link">
english_name_of_language()
</a>
</li>
<li class="md-nav__item">
<a href="#great_ai.utilities.language.is_english.is_english" class="md-nav__link">
is_english()
</a>
</li>
<li class="md-nav__item">
<a href="#great_ai.utilities.evaluate_ranking.evaluate_ranking.evaluate_ranking" class="md-nav__link">
evaluate_ranking()
</a>
</li> </li>
</ul> </ul>
@ -1463,7 +1505,8 @@ processing can be enabled with optional arguments.</p>
<td> <td>
<code>bool</code> <code>bool</code>
</td> </td>
<td><p>Filter out sentences that are not capitalised/don't end with a</p></td> <td><p>Filter out sentences that are not capitalised/don't end with a
punctuation.</p></td>
<td> <td>
<code>False</code> <code>False</code>
</td> </td>
@ -1636,12 +1679,36 @@ processing can be enabled with optional arguments.</p>
</div> </div>
<div class="doc doc-object doc-function"> <div class="doc doc-object doc-module">
<div class="doc doc-contents first"> <div class="doc doc-contents first">
<div class="doc doc-children">
<div class="doc doc-object doc-function">
<h3 id="great_ai.utilities.language.predict_language.predict_language" class="doc doc-heading">
<code class="highlight language-python"><span class="n">predict_language</span><span class="p">(</span><span class="n">text</span><span class="p">)</span></code>
<a href="#great_ai.utilities.language.predict_language.predict_language" class="headerlink" title="Permanent link">#</a></h3>
<div class="doc doc-contents ">
<p>Predict the language code from text.</p> <p>Predict the language code from text.</p>
<p>A thin wrapper over <a href="https://github.com/rspeer/langcodes">langcodes</a> for convenient <p>A thin wrapper over <a href="https://github.com/rspeer/langcodes">langcodes</a> for convenient
language tagging.</p> language tagging.</p>
@ -1755,12 +1822,194 @@ made.</p></td>
</div> </div>
<div class="doc doc-object doc-function">
</div>
</div>
</div>
<div class="doc doc-object doc-module">
<div class="doc doc-contents first"> <div class="doc doc-contents first">
<div class="doc doc-children">
<div class="doc doc-object doc-function">
<h3 id="great_ai.utilities.language.english_name_of_language.english_name_of_language" class="doc doc-heading">
<code class="highlight language-python"><span class="n">english_name_of_language</span><span class="p">(</span><span class="n">language_code</span><span class="p">)</span></code>
<a href="#great_ai.utilities.language.english_name_of_language.english_name_of_language" class="headerlink" title="Permanent link">#</a></h3>
<div class="doc doc-contents ">
<p>Human-friendly English name of language from its <code>language_code</code>.</p>
<p>A thin wrapper over <a href="https://github.com/rspeer/langcodes">langcodes</a> for convenient
language tagging.</p>
<p><strong>Examples:</strong></p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="gp">&gt;&gt;&gt; </span><span class="n">english_name_of_language</span><span class="p">(</span><span class="s1">&#39;en-US&#39;</span><span class="p">)</span>
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="go">&#39;English (United States)&#39;</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="gp">&gt;&gt;&gt; </span><span class="n">english_name_of_language</span><span class="p">(</span><span class="s1">&#39;und&#39;</span><span class="p">)</span>
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="go">&#39;Unknown language&#39;</span>
</code></pre></div>
<p><strong>Parameters:</strong></p>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Default</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>language_code</code></td>
<td>
<code><span title="typing.Optional">Optional</span>[str]</code>
</td>
<td><p>Language code, for example, returned by
<a class="autorefs autorefs-internal" href="#great_ai.utilities.language.predict_language.predict_language">great_ai.utilities.language.predict_language.predict_language</a>.</p></td>
<td>
<em>required</em>
</td>
</tr>
</tbody>
</table>
<p><strong>Returns:</strong></p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>str</code>
</td>
<td><p>English name of language.</p></td>
</tr>
</tbody>
</table>
<details class="quote">
<summary>Source code in <code>great_ai/utilities/language/english_name_of_language.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"><a href="#__codelineno-0-6"> 6</a></span>
<span class="normal"><a href="#__codelineno-0-7"> 7</a></span>
<span class="normal"><a href="#__codelineno-0-8"> 8</a></span>
<span class="normal"><a href="#__codelineno-0-9"> 9</a></span>
<span class="normal"><a href="#__codelineno-0-10">10</a></span>
<span class="normal"><a href="#__codelineno-0-11">11</a></span>
<span class="normal"><a href="#__codelineno-0-12">12</a></span>
<span class="normal"><a href="#__codelineno-0-13">13</a></span>
<span class="normal"><a href="#__codelineno-0-14">14</a></span>
<span class="normal"><a href="#__codelineno-0-15">15</a></span>
<span class="normal"><a href="#__codelineno-0-16">16</a></span>
<span class="normal"><a href="#__codelineno-0-17">17</a></span>
<span class="normal"><a href="#__codelineno-0-18">18</a></span>
<span class="normal"><a href="#__codelineno-0-19">19</a></span>
<span class="normal"><a href="#__codelineno-0-20">20</a></span>
<span class="normal"><a href="#__codelineno-0-21">21</a></span>
<span class="normal"><a href="#__codelineno-0-22">22</a></span>
<span class="normal"><a href="#__codelineno-0-23">23</a></span>
<span class="normal"><a href="#__codelineno-0-24">24</a></span>
<span class="normal"><a href="#__codelineno-0-25">25</a></span>
<span class="normal"><a href="#__codelineno-0-26">26</a></span>
<span class="normal"><a href="#__codelineno-0-27">27</a></span>
<span class="normal"><a href="#__codelineno-0-28">28</a></span>
<span class="normal"><a href="#__codelineno-0-29">29</a></span>
<span class="normal"><a href="#__codelineno-0-30">30</a></span></pre></div></td><td class="code"><div><pre><span></span><code><a id="__codelineno-0-6" name="__codelineno-0-6"></a><span class="k">def</span> <span class="nf">english_name_of_language</span><span class="p">(</span><span class="n">language_code</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
<a id="__codelineno-0-7" name="__codelineno-0-7"></a> <span class="sd">&quot;&quot;&quot;Human-friendly English name of language from its `language_code`.</span>
<a id="__codelineno-0-8" name="__codelineno-0-8"></a>
<a id="__codelineno-0-9" name="__codelineno-0-9"></a><span class="sd"> A thin wrapper over [langcodes](https://github.com/rspeer/langcodes) for convenient</span>
<a id="__codelineno-0-10" name="__codelineno-0-10"></a><span class="sd"> language tagging.</span>
<a id="__codelineno-0-11" name="__codelineno-0-11"></a>
<a id="__codelineno-0-12" name="__codelineno-0-12"></a><span class="sd"> Examples:</span>
<a id="__codelineno-0-13" name="__codelineno-0-13"></a><span class="sd"> &gt;&gt;&gt; english_name_of_language(&#39;en-US&#39;)</span>
<a id="__codelineno-0-14" name="__codelineno-0-14"></a><span class="sd"> &#39;English (United States)&#39;</span>
<a id="__codelineno-0-15" name="__codelineno-0-15"></a>
<a id="__codelineno-0-16" name="__codelineno-0-16"></a><span class="sd"> &gt;&gt;&gt; english_name_of_language(&#39;und&#39;)</span>
<a id="__codelineno-0-17" name="__codelineno-0-17"></a><span class="sd"> &#39;Unknown language&#39;</span>
<a id="__codelineno-0-18" name="__codelineno-0-18"></a>
<a id="__codelineno-0-19" name="__codelineno-0-19"></a><span class="sd"> Args:</span>
<a id="__codelineno-0-20" name="__codelineno-0-20"></a><span class="sd"> language_code: Language code, for example, returned by</span>
<a id="__codelineno-0-21" name="__codelineno-0-21"></a><span class="sd"> [great_ai.utilities.language.predict_language.predict_language][].</span>
<a id="__codelineno-0-22" name="__codelineno-0-22"></a>
<a id="__codelineno-0-23" name="__codelineno-0-23"></a><span class="sd"> Returns:</span>
<a id="__codelineno-0-24" name="__codelineno-0-24"></a><span class="sd"> English name of language.</span>
<a id="__codelineno-0-25" name="__codelineno-0-25"></a><span class="sd"> &quot;&quot;&quot;</span>
<a id="__codelineno-0-26" name="__codelineno-0-26"></a>
<a id="__codelineno-0-27" name="__codelineno-0-27"></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">language_code</span><span class="p">:</span>
<a id="__codelineno-0-28" name="__codelineno-0-28"></a> <span class="n">language_code</span> <span class="o">=</span> <span class="s2">&quot;und&quot;</span>
<a id="__codelineno-0-29" name="__codelineno-0-29"></a>
<a id="__codelineno-0-30" name="__codelineno-0-30"></a> <span class="k">return</span> <span class="n">Language</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">language_code</span><span class="p">)</span><span class="o">.</span><span class="n">display_name</span><span class="p">()</span>
</code></pre></div></td></tr></table></div>
</details>
</div>
</div>
</div>
</div>
</div>
<div class="doc doc-object doc-module">
<div class="doc doc-contents first">
<div class="doc doc-children">
<div class="doc doc-object doc-function">
<h3 id="great_ai.utilities.language.is_english.is_english" class="doc doc-heading">
<code class="highlight language-python"><span class="n">is_english</span><span class="p">(</span><span class="n">language_code</span><span class="p">)</span></code>
<a href="#great_ai.utilities.language.is_english.is_english" class="headerlink" title="Permanent link">#</a></h3>
<div class="doc doc-contents ">
<p>Decide whether the <code>language_code</code> is of an English language.</p> <p>Decide whether the <code>language_code</code> is of an English language.</p>
<p>A thin wrapper over <a href="https://github.com/rspeer/langcodes">langcodes</a> for convenient <p>A thin wrapper over <a href="https://github.com/rspeer/langcodes">langcodes</a> for convenient
language tagging.</p> language tagging.</p>
@ -1793,7 +2042,7 @@ language tagging.</p>
<code><span title="typing.Optional">Optional</span>[str]</code> <code><span title="typing.Optional">Optional</span>[str]</code>
</td> </td>
<td><p>Language code, for example, returned by <td><p>Language code, for example, returned by
`[great_ai.utilities.predict_language][].</p></td> `<a class="autorefs autorefs-internal" href="#great_ai.utilities.language.predict_language.predict_language">great_ai.utilities.language.predict_language.predict_language</a>.</p></td>
<td> <td>
<em>required</em> <em>required</em>
</td> </td>
@ -1866,7 +2115,7 @@ language tagging.</p>
<a id="__codelineno-0-21" name="__codelineno-0-21"></a> <a id="__codelineno-0-21" name="__codelineno-0-21"></a>
<a id="__codelineno-0-22" name="__codelineno-0-22"></a><span class="sd"> Args:</span> <a id="__codelineno-0-22" name="__codelineno-0-22"></a><span class="sd"> Args:</span>
<a id="__codelineno-0-23" name="__codelineno-0-23"></a><span class="sd"> language_code: Language code, for example, returned by</span> <a id="__codelineno-0-23" name="__codelineno-0-23"></a><span class="sd"> language_code: Language code, for example, returned by</span>
<a id="__codelineno-0-24" name="__codelineno-0-24"></a><span class="sd"> `[great_ai.utilities.predict_language][].</span> <a id="__codelineno-0-24" name="__codelineno-0-24"></a><span class="sd"> `[great_ai.utilities.language.predict_language.predict_language][].</span>
<a id="__codelineno-0-25" name="__codelineno-0-25"></a> <a id="__codelineno-0-25" name="__codelineno-0-25"></a>
<a id="__codelineno-0-26" name="__codelineno-0-26"></a><span class="sd"> Returns:</span> <a id="__codelineno-0-26" name="__codelineno-0-26"></a><span class="sd"> Returns:</span>
<a id="__codelineno-0-27" name="__codelineno-0-27"></a><span class="sd"> Boolean indicating whether it&#39;s English.</span> <a id="__codelineno-0-27" name="__codelineno-0-27"></a><span class="sd"> Boolean indicating whether it&#39;s English.</span>
@ -1882,120 +2131,10 @@ language tagging.</p>
</div> </div>
<div class="doc doc-object doc-function">
</div>
<div class="doc doc-contents first">
<p>Human-friendly English name of language from its <code>language_code</code>.</p>
<p>A thin wrapper over <a href="https://github.com/rspeer/langcodes">langcodes</a> for convenient
language tagging.</p>
<p><strong>Examples:</strong></p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="gp">&gt;&gt;&gt; </span><span class="n">english_name_of_language</span><span class="p">(</span><span class="s1">&#39;en-US&#39;</span><span class="p">)</span>
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="go">&#39;English (United States)&#39;</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="gp">&gt;&gt;&gt; </span><span class="n">english_name_of_language</span><span class="p">(</span><span class="s1">&#39;und&#39;</span><span class="p">)</span>
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="go">&#39;Unknown language&#39;</span>
</code></pre></div>
<p><strong>Parameters:</strong></p>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Default</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>language_code</code></td>
<td>
<code><span title="typing.Optional">Optional</span>[str]</code>
</td>
<td><p>Language code, for example, returned by
[great_ai.utilities.predict_language][].</p></td>
<td>
<em>required</em>
</td>
</tr>
</tbody>
</table>
<p><strong>Returns:</strong></p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>str</code>
</td>
<td><p>English name of language.</p></td>
</tr>
</tbody>
</table>
<details class="quote">
<summary>Source code in <code>great_ai/utilities/language/english_name_of_language.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"><a href="#__codelineno-0-6"> 6</a></span>
<span class="normal"><a href="#__codelineno-0-7"> 7</a></span>
<span class="normal"><a href="#__codelineno-0-8"> 8</a></span>
<span class="normal"><a href="#__codelineno-0-9"> 9</a></span>
<span class="normal"><a href="#__codelineno-0-10">10</a></span>
<span class="normal"><a href="#__codelineno-0-11">11</a></span>
<span class="normal"><a href="#__codelineno-0-12">12</a></span>
<span class="normal"><a href="#__codelineno-0-13">13</a></span>
<span class="normal"><a href="#__codelineno-0-14">14</a></span>
<span class="normal"><a href="#__codelineno-0-15">15</a></span>
<span class="normal"><a href="#__codelineno-0-16">16</a></span>
<span class="normal"><a href="#__codelineno-0-17">17</a></span>
<span class="normal"><a href="#__codelineno-0-18">18</a></span>
<span class="normal"><a href="#__codelineno-0-19">19</a></span>
<span class="normal"><a href="#__codelineno-0-20">20</a></span>
<span class="normal"><a href="#__codelineno-0-21">21</a></span>
<span class="normal"><a href="#__codelineno-0-22">22</a></span>
<span class="normal"><a href="#__codelineno-0-23">23</a></span>
<span class="normal"><a href="#__codelineno-0-24">24</a></span>
<span class="normal"><a href="#__codelineno-0-25">25</a></span>
<span class="normal"><a href="#__codelineno-0-26">26</a></span>
<span class="normal"><a href="#__codelineno-0-27">27</a></span>
<span class="normal"><a href="#__codelineno-0-28">28</a></span>
<span class="normal"><a href="#__codelineno-0-29">29</a></span>
<span class="normal"><a href="#__codelineno-0-30">30</a></span></pre></div></td><td class="code"><div><pre><span></span><code><a id="__codelineno-0-6" name="__codelineno-0-6"></a><span class="k">def</span> <span class="nf">english_name_of_language</span><span class="p">(</span><span class="n">language_code</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
<a id="__codelineno-0-7" name="__codelineno-0-7"></a> <span class="sd">&quot;&quot;&quot;Human-friendly English name of language from its `language_code`.</span>
<a id="__codelineno-0-8" name="__codelineno-0-8"></a>
<a id="__codelineno-0-9" name="__codelineno-0-9"></a><span class="sd"> A thin wrapper over [langcodes](https://github.com/rspeer/langcodes) for convenient</span>
<a id="__codelineno-0-10" name="__codelineno-0-10"></a><span class="sd"> language tagging.</span>
<a id="__codelineno-0-11" name="__codelineno-0-11"></a>
<a id="__codelineno-0-12" name="__codelineno-0-12"></a><span class="sd"> Examples:</span>
<a id="__codelineno-0-13" name="__codelineno-0-13"></a><span class="sd"> &gt;&gt;&gt; english_name_of_language(&#39;en-US&#39;)</span>
<a id="__codelineno-0-14" name="__codelineno-0-14"></a><span class="sd"> &#39;English (United States)&#39;</span>
<a id="__codelineno-0-15" name="__codelineno-0-15"></a>
<a id="__codelineno-0-16" name="__codelineno-0-16"></a><span class="sd"> &gt;&gt;&gt; english_name_of_language(&#39;und&#39;)</span>
<a id="__codelineno-0-17" name="__codelineno-0-17"></a><span class="sd"> &#39;Unknown language&#39;</span>
<a id="__codelineno-0-18" name="__codelineno-0-18"></a>
<a id="__codelineno-0-19" name="__codelineno-0-19"></a><span class="sd"> Args:</span>
<a id="__codelineno-0-20" name="__codelineno-0-20"></a><span class="sd"> language_code: Language code, for example, returned by</span>
<a id="__codelineno-0-21" name="__codelineno-0-21"></a><span class="sd"> [great_ai.utilities.predict_language][].</span>
<a id="__codelineno-0-22" name="__codelineno-0-22"></a>
<a id="__codelineno-0-23" name="__codelineno-0-23"></a><span class="sd"> Returns:</span>
<a id="__codelineno-0-24" name="__codelineno-0-24"></a><span class="sd"> English name of language.</span>
<a id="__codelineno-0-25" name="__codelineno-0-25"></a><span class="sd"> &quot;&quot;&quot;</span>
<a id="__codelineno-0-26" name="__codelineno-0-26"></a>
<a id="__codelineno-0-27" name="__codelineno-0-27"></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">language_code</span><span class="p">:</span>
<a id="__codelineno-0-28" name="__codelineno-0-28"></a> <span class="n">language_code</span> <span class="o">=</span> <span class="s2">&quot;und&quot;</span>
<a id="__codelineno-0-29" name="__codelineno-0-29"></a>
<a id="__codelineno-0-30" name="__codelineno-0-30"></a> <span class="k">return</span> <span class="n">Language</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">language_code</span><span class="p">)</span><span class="o">.</span><span class="n">display_name</span><span class="p">()</span>
</code></pre></div></td></tr></table></div>
</details>
</div> </div>
</div> </div>
@ -2018,6 +2157,338 @@ language tagging.</p>
<div class="doc doc-object doc-function">
<h3 id="great_ai.utilities.evaluate_ranking.evaluate_ranking.evaluate_ranking" class="doc doc-heading">
<code class="highlight language-python"><span class="n">evaluate_ranking</span><span class="p">(</span><span class="n">expected</span><span class="p">,</span> <span class="n">actual_scores</span><span class="p">,</span> <span class="n">target_recall</span><span class="p">,</span> <span class="n">title</span><span class="o">=</span><span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="n">disable_interpolation</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">axes</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">output_svg</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">reverse_order</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">plot</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span></code>
<a href="#great_ai.utilities.evaluate_ranking.evaluate_ranking.evaluate_ranking" class="headerlink" title="Permanent link">#</a></h3>
<div class="doc doc-contents ">
<p>Render the Precision-Recall curve of a ranking.</p>
<p>And improved version of scikit-learn's <a href="https://scikit-learn.org/stable/auto_examples/model_selection/plot_precision_recall.html#sphx-glr-auto-examples-model-selection-plot-precision-recall-py">PR-curve</a></p>
<p><strong>Parameters:</strong></p>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Default</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>expected</code></td>
<td>
<code><span title="typing.List">List</span>[<span title="great_ai.utilities.evaluate_ranking.evaluate_ranking.T">T</span>]</code>
</td>
<td><p>Expected ordering of the elements
(rank if it's an integer, alphabetical if a string)</p></td>
<td>
<em>required</em>
</td>
</tr>
<tr>
<td><code>actual_scores</code></td>
<td>
<code><span title="typing.List">List</span>[float]</code>
</td>
<td><p>Actual ranking scores (need not be on the same scale as
<code>expected</code>)</p></td>
<td>
<em>required</em>
</td>
</tr>
<tr>
<td><code>title</code></td>
<td>
<code><span title="typing.Optional">Optional</span>[str]</code>
</td>
<td><p>Title of the plot.</p></td>
<td>
<code>&#39;&#39;</code>
</td>
</tr>
<tr>
<td><code>disable_interpolation</code></td>
<td>
<code>bool</code>
</td>
<td><p>Do not interpolate.</p></td>
<td>
<code>False</code>
</td>
</tr>
<tr>
<td><code>axes</code></td>
<td>
<code><span title="typing.Optional">Optional</span>[<span title="matplotlib.pyplot">plt</span>.<span title="matplotlib.pyplot.Axes">Axes</span>]</code>
</td>
<td><p>Matplotlib axes for ploting inside a subplot.</p></td>
<td>
<code>None</code>
</td>
</tr>
<tr>
<td><code>output_svg</code></td>
<td>
<code><span title="typing.Optional">Optional</span>[<span title="pathlib.Path">Path</span>]</code>
</td>
<td><p>If specified, save the chart as an svg to the given Path.</p></td>
<td>
<code>None</code>
</td>
</tr>
<tr>
<td><code>reverse_order</code></td>
<td>
<code>bool</code>
</td>
<td><p>Reverse the ranking specified by <code>expected</code>.</p></td>
<td>
<code>False</code>
</td>
</tr>
<tr>
<td><code>plot</code></td>
<td>
<code>bool</code>
</td>
<td><p>Display a plot on the screen.</p></td>
<td>
<code>True</code>
</td>
</tr>
</tbody>
</table>
<p><strong>Returns:</strong></p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code><span title="typing.Dict">Dict</span>[<span title="great_ai.utilities.evaluate_ranking.evaluate_ranking.T">T</span>, float]</code>
</td>
<td><p>Precision values at given recall.</p></td>
</tr>
</tbody>
</table>
<details class="quote">
<summary>Source code in <code>great_ai/utilities/evaluate_ranking/evaluate_ranking.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"><a href="#__codelineno-0-15"> 15</a></span>
<span class="normal"><a href="#__codelineno-0-16"> 16</a></span>
<span class="normal"><a href="#__codelineno-0-17"> 17</a></span>
<span class="normal"><a href="#__codelineno-0-18"> 18</a></span>
<span class="normal"><a href="#__codelineno-0-19"> 19</a></span>
<span class="normal"><a href="#__codelineno-0-20"> 20</a></span>
<span class="normal"><a href="#__codelineno-0-21"> 21</a></span>
<span class="normal"><a href="#__codelineno-0-22"> 22</a></span>
<span class="normal"><a href="#__codelineno-0-23"> 23</a></span>
<span class="normal"><a href="#__codelineno-0-24"> 24</a></span>
<span class="normal"><a href="#__codelineno-0-25"> 25</a></span>
<span class="normal"><a href="#__codelineno-0-26"> 26</a></span>
<span class="normal"><a href="#__codelineno-0-27"> 27</a></span>
<span class="normal"><a href="#__codelineno-0-28"> 28</a></span>
<span class="normal"><a href="#__codelineno-0-29"> 29</a></span>
<span class="normal"><a href="#__codelineno-0-30"> 30</a></span>
<span class="normal"><a href="#__codelineno-0-31"> 31</a></span>
<span class="normal"><a href="#__codelineno-0-32"> 32</a></span>
<span class="normal"><a href="#__codelineno-0-33"> 33</a></span>
<span class="normal"><a href="#__codelineno-0-34"> 34</a></span>
<span class="normal"><a href="#__codelineno-0-35"> 35</a></span>
<span class="normal"><a href="#__codelineno-0-36"> 36</a></span>
<span class="normal"><a href="#__codelineno-0-37"> 37</a></span>
<span class="normal"><a href="#__codelineno-0-38"> 38</a></span>
<span class="normal"><a href="#__codelineno-0-39"> 39</a></span>
<span class="normal"><a href="#__codelineno-0-40"> 40</a></span>
<span class="normal"><a href="#__codelineno-0-41"> 41</a></span>
<span class="normal"><a href="#__codelineno-0-42"> 42</a></span>
<span class="normal"><a href="#__codelineno-0-43"> 43</a></span>
<span class="normal"><a href="#__codelineno-0-44"> 44</a></span>
<span class="normal"><a href="#__codelineno-0-45"> 45</a></span>
<span class="normal"><a href="#__codelineno-0-46"> 46</a></span>
<span class="normal"><a href="#__codelineno-0-47"> 47</a></span>
<span class="normal"><a href="#__codelineno-0-48"> 48</a></span>
<span class="normal"><a href="#__codelineno-0-49"> 49</a></span>
<span class="normal"><a href="#__codelineno-0-50"> 50</a></span>
<span class="normal"><a href="#__codelineno-0-51"> 51</a></span>
<span class="normal"><a href="#__codelineno-0-52"> 52</a></span>
<span class="normal"><a href="#__codelineno-0-53"> 53</a></span>
<span class="normal"><a href="#__codelineno-0-54"> 54</a></span>
<span class="normal"><a href="#__codelineno-0-55"> 55</a></span>
<span class="normal"><a href="#__codelineno-0-56"> 56</a></span>
<span class="normal"><a href="#__codelineno-0-57"> 57</a></span>
<span class="normal"><a href="#__codelineno-0-58"> 58</a></span>
<span class="normal"><a href="#__codelineno-0-59"> 59</a></span>
<span class="normal"><a href="#__codelineno-0-60"> 60</a></span>
<span class="normal"><a href="#__codelineno-0-61"> 61</a></span>
<span class="normal"><a href="#__codelineno-0-62"> 62</a></span>
<span class="normal"><a href="#__codelineno-0-63"> 63</a></span>
<span class="normal"><a href="#__codelineno-0-64"> 64</a></span>
<span class="normal"><a href="#__codelineno-0-65"> 65</a></span>
<span class="normal"><a href="#__codelineno-0-66"> 66</a></span>
<span class="normal"><a href="#__codelineno-0-67"> 67</a></span>
<span class="normal"><a href="#__codelineno-0-68"> 68</a></span>
<span class="normal"><a href="#__codelineno-0-69"> 69</a></span>
<span class="normal"><a href="#__codelineno-0-70"> 70</a></span>
<span class="normal"><a href="#__codelineno-0-71"> 71</a></span>
<span class="normal"><a href="#__codelineno-0-72"> 72</a></span>
<span class="normal"><a href="#__codelineno-0-73"> 73</a></span>
<span class="normal"><a href="#__codelineno-0-74"> 74</a></span>
<span class="normal"><a href="#__codelineno-0-75"> 75</a></span>
<span class="normal"><a href="#__codelineno-0-76"> 76</a></span>
<span class="normal"><a href="#__codelineno-0-77"> 77</a></span>
<span class="normal"><a href="#__codelineno-0-78"> 78</a></span>
<span class="normal"><a href="#__codelineno-0-79"> 79</a></span>
<span class="normal"><a href="#__codelineno-0-80"> 80</a></span>
<span class="normal"><a href="#__codelineno-0-81"> 81</a></span>
<span class="normal"><a href="#__codelineno-0-82"> 82</a></span>
<span class="normal"><a href="#__codelineno-0-83"> 83</a></span>
<span class="normal"><a href="#__codelineno-0-84"> 84</a></span>
<span class="normal"><a href="#__codelineno-0-85"> 85</a></span>
<span class="normal"><a href="#__codelineno-0-86"> 86</a></span>
<span class="normal"><a href="#__codelineno-0-87"> 87</a></span>
<span class="normal"><a href="#__codelineno-0-88"> 88</a></span>
<span class="normal"><a href="#__codelineno-0-89"> 89</a></span>
<span class="normal"><a href="#__codelineno-0-90"> 90</a></span>
<span class="normal"><a href="#__codelineno-0-91"> 91</a></span>
<span class="normal"><a href="#__codelineno-0-92"> 92</a></span>
<span class="normal"><a href="#__codelineno-0-93"> 93</a></span>
<span class="normal"><a href="#__codelineno-0-94"> 94</a></span>
<span class="normal"><a href="#__codelineno-0-95"> 95</a></span>
<span class="normal"><a href="#__codelineno-0-96"> 96</a></span>
<span class="normal"><a href="#__codelineno-0-97"> 97</a></span>
<span class="normal"><a href="#__codelineno-0-98"> 98</a></span>
<span class="normal"><a href="#__codelineno-0-99"> 99</a></span>
<span class="normal"><a href="#__codelineno-0-100">100</a></span>
<span class="normal"><a href="#__codelineno-0-101">101</a></span>
<span class="normal"><a href="#__codelineno-0-102">102</a></span>
<span class="normal"><a href="#__codelineno-0-103">103</a></span>
<span class="normal"><a href="#__codelineno-0-104">104</a></span>
<span class="normal"><a href="#__codelineno-0-105">105</a></span>
<span class="normal"><a href="#__codelineno-0-106">106</a></span>
<span class="normal"><a href="#__codelineno-0-107">107</a></span>
<span class="normal"><a href="#__codelineno-0-108">108</a></span>
<span class="normal"><a href="#__codelineno-0-109">109</a></span>
<span class="normal"><a href="#__codelineno-0-110">110</a></span>
<span class="normal"><a href="#__codelineno-0-111">111</a></span>
<span class="normal"><a href="#__codelineno-0-112">112</a></span></pre></div></td><td class="code"><div><pre><span></span><code><a id="__codelineno-0-15" name="__codelineno-0-15"></a><span class="k">def</span> <span class="nf">evaluate_ranking</span><span class="p">(</span>
<a id="__codelineno-0-16" name="__codelineno-0-16"></a> <span class="n">expected</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">T</span><span class="p">],</span>
<a id="__codelineno-0-17" name="__codelineno-0-17"></a> <span class="n">actual_scores</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="nb">float</span><span class="p">],</span>
<a id="__codelineno-0-18" name="__codelineno-0-18"></a> <span class="n">target_recall</span><span class="p">:</span> <span class="nb">float</span><span class="p">,</span>
<a id="__codelineno-0-19" name="__codelineno-0-19"></a> <span class="n">title</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span><span class="p">,</span>
<a id="__codelineno-0-20" name="__codelineno-0-20"></a> <span class="n">disable_interpolation</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
<a id="__codelineno-0-21" name="__codelineno-0-21"></a> <span class="n">axes</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">plt</span><span class="o">.</span><span class="n">Axes</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
<a id="__codelineno-0-22" name="__codelineno-0-22"></a> <span class="n">output_svg</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Path</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
<a id="__codelineno-0-23" name="__codelineno-0-23"></a> <span class="n">reverse_order</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
<a id="__codelineno-0-24" name="__codelineno-0-24"></a> <span class="n">plot</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
<a id="__codelineno-0-25" name="__codelineno-0-25"></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Dict</span><span class="p">[</span><span class="n">T</span><span class="p">,</span> <span class="nb">float</span><span class="p">]:</span>
<a id="__codelineno-0-26" name="__codelineno-0-26"></a> <span class="sd">&quot;&quot;&quot;Render the Precision-Recall curve of a ranking.</span>
<a id="__codelineno-0-27" name="__codelineno-0-27"></a>
<a id="__codelineno-0-28" name="__codelineno-0-28"></a><span class="sd"> And improved version of scikit-learn&#39;s [PR-curve](https://scikit-learn.org/stable/auto_examples/model_selection/plot_precision_recall.html#sphx-glr-auto-examples-model-selection-plot-precision-recall-py)</span>
<a id="__codelineno-0-29" name="__codelineno-0-29"></a>
<a id="__codelineno-0-30" name="__codelineno-0-30"></a><span class="sd"> Args:</span>
<a id="__codelineno-0-31" name="__codelineno-0-31"></a><span class="sd"> expected: Expected ordering of the elements</span>
<a id="__codelineno-0-32" name="__codelineno-0-32"></a><span class="sd"> (rank if it&#39;s an integer, alphabetical if a string)</span>
<a id="__codelineno-0-33" name="__codelineno-0-33"></a><span class="sd"> actual_scores: Actual ranking scores (need not be on the same scale as</span>
<a id="__codelineno-0-34" name="__codelineno-0-34"></a><span class="sd"> `expected`)</span>
<a id="__codelineno-0-35" name="__codelineno-0-35"></a><span class="sd"> title: Title of the plot.</span>
<a id="__codelineno-0-36" name="__codelineno-0-36"></a><span class="sd"> disable_interpolation: Do not interpolate.</span>
<a id="__codelineno-0-37" name="__codelineno-0-37"></a><span class="sd"> axes: Matplotlib axes for ploting inside a subplot.</span>
<a id="__codelineno-0-38" name="__codelineno-0-38"></a><span class="sd"> output_svg: If specified, save the chart as an svg to the given Path.</span>
<a id="__codelineno-0-39" name="__codelineno-0-39"></a><span class="sd"> reverse_order: Reverse the ranking specified by `expected`.</span>
<a id="__codelineno-0-40" name="__codelineno-0-40"></a><span class="sd"> plot: Display a plot on the screen.</span>
<a id="__codelineno-0-41" name="__codelineno-0-41"></a>
<a id="__codelineno-0-42" name="__codelineno-0-42"></a><span class="sd"> Returns:</span>
<a id="__codelineno-0-43" name="__codelineno-0-43"></a><span class="sd"> Precision values at given recall.</span>
<a id="__codelineno-0-44" name="__codelineno-0-44"></a><span class="sd"> &quot;&quot;&quot;</span>
<a id="__codelineno-0-45" name="__codelineno-0-45"></a>
<a id="__codelineno-0-46" name="__codelineno-0-46"></a> <span class="k">assert</span> <span class="mi">0</span> <span class="o">&lt;=</span> <span class="n">target_recall</span> <span class="o">&lt;=</span> <span class="mi">1</span>
<a id="__codelineno-0-47" name="__codelineno-0-47"></a>
<a id="__codelineno-0-48" name="__codelineno-0-48"></a> <span class="k">if</span> <span class="n">plot</span> <span class="ow">and</span> <span class="n">axes</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<a id="__codelineno-0-49" name="__codelineno-0-49"></a> <span class="n">fig</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">20</span><span class="p">,</span> <span class="mi">10</span><span class="p">))</span>
<a id="__codelineno-0-50" name="__codelineno-0-50"></a> <span class="n">fig</span><span class="o">.</span><span class="n">patch</span><span class="o">.</span><span class="n">set_facecolor</span><span class="p">(</span><span class="s2">&quot;white&quot;</span><span class="p">)</span>
<a id="__codelineno-0-51" name="__codelineno-0-51"></a> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">axes</span><span class="p">()</span>
<a id="__codelineno-0-52" name="__codelineno-0-52"></a> <span class="k">else</span><span class="p">:</span>
<a id="__codelineno-0-53" name="__codelineno-0-53"></a> <span class="n">ax</span> <span class="o">=</span> <span class="n">axes</span>
<a id="__codelineno-0-54" name="__codelineno-0-54"></a>
<a id="__codelineno-0-55" name="__codelineno-0-55"></a> <span class="n">classes</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">unique</span><span class="p">(</span><span class="n">expected</span><span class="p">),</span> <span class="n">reverse</span><span class="o">=</span><span class="n">reverse_order</span><span class="p">)</span>
<a id="__codelineno-0-56" name="__codelineno-0-56"></a> <span class="n">str_classes</span> <span class="o">=</span> <span class="p">[</span><span class="nb">str</span><span class="p">(</span><span class="n">c</span><span class="p">)</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">classes</span><span class="p">]</span>
<a id="__codelineno-0-57" name="__codelineno-0-57"></a>
<a id="__codelineno-0-58" name="__codelineno-0-58"></a> <span class="k">with</span> <span class="n">matplotlib</span><span class="o">.</span><span class="n">rc_context</span><span class="p">({</span><span class="s2">&quot;font.size&quot;</span><span class="p">:</span> <span class="mi">20</span><span class="p">}):</span>
<a id="__codelineno-0-59" name="__codelineno-0-59"></a> <span class="k">if</span> <span class="n">plot</span><span class="p">:</span>
<a id="__codelineno-0-60" name="__codelineno-0-60"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_xmargin</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<a id="__codelineno-0-61" name="__codelineno-0-61"></a>
<a id="__codelineno-0-62" name="__codelineno-0-62"></a> <span class="n">draw_f1_iso_lines</span><span class="p">(</span><span class="n">axes</span><span class="o">=</span><span class="n">ax</span><span class="p">)</span>
<a id="__codelineno-0-63" name="__codelineno-0-63"></a>
<a id="__codelineno-0-64" name="__codelineno-0-64"></a> <span class="n">results</span><span class="p">:</span> <span class="n">Dict</span><span class="p">[</span><span class="n">T</span><span class="p">,</span> <span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
<a id="__codelineno-0-65" name="__codelineno-0-65"></a> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">classes</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">):</span>
<a id="__codelineno-0-66" name="__codelineno-0-66"></a> <span class="n">binarized_expected</span> <span class="o">=</span> <span class="p">[</span>
<a id="__codelineno-0-67" name="__codelineno-0-67"></a> <span class="p">(</span><span class="n">v</span> <span class="o">&lt;</span> <span class="n">classes</span><span class="p">[</span><span class="n">i</span><span class="p">])</span> <span class="k">if</span> <span class="n">reverse_order</span> <span class="k">else</span> <span class="p">(</span><span class="n">v</span> <span class="o">&gt;</span> <span class="n">classes</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
<a id="__codelineno-0-68" name="__codelineno-0-68"></a> <span class="k">for</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">expected</span>
<a id="__codelineno-0-69" name="__codelineno-0-69"></a> <span class="p">]</span>
<a id="__codelineno-0-70" name="__codelineno-0-70"></a> <span class="n">precision</span><span class="p">,</span> <span class="n">recall</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="n">precision_recall_curve</span><span class="p">(</span>
<a id="__codelineno-0-71" name="__codelineno-0-71"></a> <span class="n">binarized_expected</span><span class="p">,</span> <span class="n">actual_scores</span>
<a id="__codelineno-0-72" name="__codelineno-0-72"></a> <span class="p">)</span>
<a id="__codelineno-0-73" name="__codelineno-0-73"></a>
<a id="__codelineno-0-74" name="__codelineno-0-74"></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">disable_interpolation</span><span class="p">:</span>
<a id="__codelineno-0-75" name="__codelineno-0-75"></a> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">precision</span><span class="p">)):</span>
<a id="__codelineno-0-76" name="__codelineno-0-76"></a> <span class="n">precision</span><span class="p">[</span><span class="n">j</span><span class="p">]</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">precision</span><span class="p">[</span><span class="n">j</span> <span class="o">-</span> <span class="mi">1</span><span class="p">],</span> <span class="n">precision</span><span class="p">[</span><span class="n">j</span><span class="p">])</span>
<a id="__codelineno-0-77" name="__codelineno-0-77"></a>
<a id="__codelineno-0-78" name="__codelineno-0-78"></a> <span class="n">closest_recall_index</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">argmin</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">abs</span><span class="p">(</span><span class="n">recall</span> <span class="o">-</span> <span class="n">target_recall</span><span class="p">))</span>
<a id="__codelineno-0-79" name="__codelineno-0-79"></a> <span class="n">precision_at_closest_recall</span> <span class="o">=</span> <span class="n">precision</span><span class="p">[</span><span class="n">closest_recall_index</span><span class="p">]</span>
<a id="__codelineno-0-80" name="__codelineno-0-80"></a> <span class="n">average_precision</span> <span class="o">=</span> <span class="n">average_precision_score</span><span class="p">(</span>
<a id="__codelineno-0-81" name="__codelineno-0-81"></a> <span class="n">binarized_expected</span><span class="p">,</span> <span class="n">actual_scores</span>
<a id="__codelineno-0-82" name="__codelineno-0-82"></a> <span class="p">)</span>
<a id="__codelineno-0-83" name="__codelineno-0-83"></a> <span class="n">results</span><span class="p">[</span><span class="n">classes</span><span class="p">[</span><span class="n">i</span><span class="p">]]</span> <span class="o">=</span> <span class="n">precision_at_closest_recall</span>
<a id="__codelineno-0-84" name="__codelineno-0-84"></a>
<a id="__codelineno-0-85" name="__codelineno-0-85"></a> <span class="k">if</span> <span class="n">plot</span><span class="p">:</span>
<a id="__codelineno-0-86" name="__codelineno-0-86"></a> <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span>
<a id="__codelineno-0-87" name="__codelineno-0-87"></a> <span class="n">recall</span><span class="p">,</span>
<a id="__codelineno-0-88" name="__codelineno-0-88"></a> <span class="n">precision</span><span class="p">,</span>
<a id="__codelineno-0-89" name="__codelineno-0-89"></a> <span class="n">label</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="s1">&#39;|&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">[:</span><span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">])</span><span class="si">}</span><span class="s2"> ↔ </span><span class="si">{</span><span class="s1">&#39;|&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">[</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">:])</span><span class="si">}</span><span class="s2"> (P@</span><span class="si">{</span><span class="n">target_recall</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">=</span><span class="si">{</span><span class="n">precision_at_closest_recall</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">, AP=</span><span class="si">{</span><span class="n">average_precision</span><span class="si">:</span><span class="s2">.2f</span><span class="si">}</span><span class="s2">)&quot;</span><span class="p">,</span>
<a id="__codelineno-0-90" name="__codelineno-0-90"></a> <span class="p">)</span>
<a id="__codelineno-0-91" name="__codelineno-0-91"></a>
<a id="__codelineno-0-92" name="__codelineno-0-92"></a> <span class="k">if</span> <span class="n">plot</span><span class="p">:</span>
<a id="__codelineno-0-93" name="__codelineno-0-93"></a> <span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="s2">&quot;upper right&quot;</span><span class="p">)</span>
<a id="__codelineno-0-94" name="__codelineno-0-94"></a> <span class="n">ax</span><span class="o">.</span><span class="n">axvline</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">target_recall</span><span class="p">,</span> <span class="n">linestyle</span><span class="o">=</span><span class="s2">&quot;--&quot;</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="s2">&quot;#55c6bb&quot;</span><span class="p">,</span> <span class="n">linewidth</span><span class="o">=</span><span class="mf">2.0</span><span class="p">)</span>
<a id="__codelineno-0-95" name="__codelineno-0-95"></a>
<a id="__codelineno-0-96" name="__codelineno-0-96"></a> <span class="k">if</span> <span class="n">title</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<a id="__codelineno-0-97" name="__codelineno-0-97"></a> <span class="n">title</span> <span class="o">=</span> <span class="s2">&quot;Ranking evaluation&quot;</span>
<a id="__codelineno-0-98" name="__codelineno-0-98"></a>
<a id="__codelineno-0-99" name="__codelineno-0-99"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="sa">f</span><span class="s1">&#39;</span><span class="si">{</span><span class="n">title</span><span class="si">}</span><span class="s1"> (</span><span class="si">{</span><span class="s2">&quot; &lt; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">str_classes</span><span class="p">)</span><span class="si">}</span><span class="s1">)&#39;</span><span class="p">,</span> <span class="n">pad</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
<a id="__codelineno-0-100" name="__codelineno-0-100"></a>
<a id="__codelineno-0-101" name="__codelineno-0-101"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s2">&quot;Recall&quot;</span><span class="p">)</span>
<a id="__codelineno-0-102" name="__codelineno-0-102"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_ylabel</span><span class="p">(</span><span class="s2">&quot;Precision&quot;</span><span class="p">)</span>
<a id="__codelineno-0-103" name="__codelineno-0-103"></a>
<a id="__codelineno-0-104" name="__codelineno-0-104"></a> <span class="n">ax</span><span class="o">.</span><span class="n">set_xticks</span><span class="p">([</span><span class="n">target_recall</span><span class="p">]</span> <span class="o">+</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">ax</span><span class="o">.</span><span class="n">get_xticks</span><span class="p">()))</span>
<a id="__codelineno-0-105" name="__codelineno-0-105"></a>
<a id="__codelineno-0-106" name="__codelineno-0-106"></a> <span class="k">if</span> <span class="n">plot</span> <span class="ow">and</span> <span class="n">output_svg</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<a id="__codelineno-0-107" name="__codelineno-0-107"></a> <span class="k">if</span> <span class="n">axes</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<a id="__codelineno-0-108" name="__codelineno-0-108"></a> <span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<a id="__codelineno-0-109" name="__codelineno-0-109"></a> <span class="k">elif</span> <span class="n">output_svg</span><span class="p">:</span>
<a id="__codelineno-0-110" name="__codelineno-0-110"></a> <span class="n">plt</span><span class="o">.</span><span class="n">savefig</span><span class="p">(</span><span class="n">output_svg</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="s2">&quot;svg&quot;</span><span class="p">)</span>
<a id="__codelineno-0-111" name="__codelineno-0-111"></a>
<a id="__codelineno-0-112" name="__codelineno-0-112"></a> <span class="k">return</span> <span class="n">results</span>
</code></pre></div></td></tr></table></div>
</details>
</div>
</div>
</div> </div>
@ -2999,7 +3470,7 @@ thread and <code>ignore_exceptions=False</code>.</p></td>
</div> </div>
</div><h2 id="composable-parallel-processing">Composable parallel processing<a class="headerlink" href="#composable-parallel-processing" title="Permanent link">#</a></h2> </div><h2 id="composable-parallel-processing">Composable parallel processing<a class="headerlink" href="#composable-parallel-processing" title="Permanent link">#</a></h2>
<p>Because both <a class="autorefs autorefs-internal" href="#great_ai.utilities.threaded_parallel_map">threaded_parallel_map</a> and [parallel_map][great_ai.utilities.parallel_map.parallel_map] have a streaming interface, it is easy to compose them and end up with, for example, a process for each CPU core with its own thread-pool or event-loop. Longer pipelines are also easy to imagine. The chunking methods help in these compositions.</p> <p>Because both <a class="autorefs autorefs-internal" href="#great_ai.utilities.threaded_parallel_map">threaded_parallel_map</a> and <a class="autorefs autorefs-internal" href="#great_ai.utilities.parallel_map.parallel_map.parallel_map">parallel_map</a> have a streaming interface, it is easy to compose them and end up with, for example, a process for each CPU core with its own thread-pool or event-loop. Longer pipelines are also easy to imagine. The chunking methods help in these compositions.</p>
<p>For more info, check-out <a href="/how-to-guides/scraping">the scraping guide</a>.</p> <p>For more info, check-out <a href="/how-to-guides/scraping">the scraping guide</a>.</p>
@ -3953,7 +4424,7 @@ the system and <code>ignore_missing=False</code>.</p></td>
<small> <small>
Last update: Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 11, 2022</span> <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 13, 2022</span>
</small> </small>

10
reference/utilities/utilities.md
View file

@ -10,10 +10,10 @@ Well-tested tools that can be used in production with confidence. The toolbox of
::: great_ai.utilities.clean ::: great_ai.utilities.clean
::: great_ai.utilities.get_sentences ::: great_ai.utilities.get_sentences
::: great_ai.utilities.predict_language ::: great_ai.utilities.language.predict_language
::: great_ai.utilities.is_english ::: great_ai.utilities.language.english_name_of_language
::: great_ai.utilities.english_name_of_language ::: great_ai.utilities.language.is_english
::: great_ai.utilities.evaluate_ranking ::: great_ai.utilities.evaluate_ranking.evaluate_ranking
## Parallel processing ## Parallel processing
@ -29,7 +29,7 @@ Multiprocessing and multithreading-based parallelism with support for `async` fu
## Composable parallel processing ## Composable parallel processing
Because both [threaded_parallel_map][great_ai.utilities.parallel_map.threaded_parallel_map.threaded_parallel_map] and [parallel_map][great_ai.utilities.parallel_map.parallel_map] have a streaming interface, it is easy to compose them and end up with, for example, a process for each CPU core with its own thread-pool or event-loop. Longer pipelines are also easy to imagine. The chunking methods help in these compositions. Because both [threaded_parallel_map][great_ai.utilities.parallel_map.threaded_parallel_map.threaded_parallel_map] and [parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map] have a streaming interface, it is easy to compose them and end up with, for example, a process for each CPU core with its own thread-pool or event-loop. Longer pipelines are also easy to imagine. The chunking methods help in these compositions.
For more info, check-out [the scraping guide](/how-to-guides/scraping). For more info, check-out [the scraping guide](/how-to-guides/scraping).

14
reference/views/index.html
View file

@ -457,20 +457,6 @@
<li class="md-nav__item">
<a href="../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

2
search/search_index.json
View file

File diff suppressed because one or more lines are too long

5
sitemap.xml
View file

@ -120,9 +120,4 @@
<lastmod>2022-07-13</lastmod> <lastmod>2022-07-13</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url>
<loc>None</loc>
<lastmod>2022-07-13</lastmod>
<changefreq>daily</changefreq>
</url>
</urlset> </urlset>

BIN
sitemap.xml.gz
View file

Binary file not shown.

14
tutorial/deploy/index.html
View file

@ -500,20 +500,6 @@
<li class="md-nav__item">
<a href="../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
tutorial/index.html
View file

@ -534,20 +534,6 @@
<li class="md-nav__item">
<a href="../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>

14
tutorial/train/index.html
View file

@ -500,20 +500,6 @@
<li class="md-nav__item">
<a href="../../how-to-guides/scraping/" class="md-nav__link">
Scraping
</a>
</li>
</ul> </ul>
</nav> </nav>
</li> </li>