andras/great-ai
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 4 Packages Wiki Activity Actions

Deployed 85d079f with MkDocs version: 1.3.1

Browse source
This commit is contained in:
2022-07-29 07:52:03 +00:00
parent 95fdd502a9
commit 8a093ac9d2
7 changed files with 640 additions and 640 deletions
Download patch file Download diff file Expand all files Collapse all files

142
reference/index.html
View file

@ -2834,7 +2834,7 @@ decorating the same function with <code>use_model</code> multiple times.</p>
<tr>
<td><code>version</code></td>
<td>
<code><span title="typing.Union">Union</span>[int, <span title="typing.Literal">Literal</span>[&#39;latest&#39;]]</code>
<code><span title="typing.Union">Union</span>[int, <span title="typing_extensions.Literal">Literal</span>[latest]]</code>
</td>
<td><p>The model's version as stored by the LargeFile implementation.</p></td>
<td>
@ -2874,7 +2874,18 @@ decorating the same function with <code>use_model</code> multiple times.</p>
<details class="quote">
<summary>Source code in <code>great_ai/models/use_model.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"><a href="#__codelineno-0-27">27</a></span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"><a href="#__codelineno-0-16">16</a></span>
<span class="normal"><a href="#__codelineno-0-17">17</a></span>
<span class="normal"><a href="#__codelineno-0-18">18</a></span>
<span class="normal"><a href="#__codelineno-0-19">19</a></span>
<span class="normal"><a href="#__codelineno-0-20">20</a></span>
<span class="normal"><a href="#__codelineno-0-21">21</a></span>
<span class="normal"><a href="#__codelineno-0-22">22</a></span>
<span class="normal"><a href="#__codelineno-0-23">23</a></span>
<span class="normal"><a href="#__codelineno-0-24">24</a></span>
<span class="normal"><a href="#__codelineno-0-25">25</a></span>
<span class="normal"><a href="#__codelineno-0-26">26</a></span>
<span class="normal"><a href="#__codelineno-0-27">27</a></span>
<span class="normal"><a href="#__codelineno-0-28">28</a></span>
<span class="normal"><a href="#__codelineno-0-29">29</a></span>
<span class="normal"><a href="#__codelineno-0-30">30</a></span>
@ -2921,76 +2932,65 @@ decorating the same function with <code>use_model</code> multiple times.</p>
<span class="normal"><a href="#__codelineno-0-71">71</a></span>
<span class="normal"><a href="#__codelineno-0-72">72</a></span>
<span class="normal"><a href="#__codelineno-0-73">73</a></span>
<span class="normal"><a href="#__codelineno-0-74">74</a></span>
<span class="normal"><a href="#__codelineno-0-75">75</a></span>
<span class="normal"><a href="#__codelineno-0-76">76</a></span>
<span class="normal"><a href="#__codelineno-0-77">77</a></span>
<span class="normal"><a href="#__codelineno-0-78">78</a></span>
<span class="normal"><a href="#__codelineno-0-79">79</a></span>
<span class="normal"><a href="#__codelineno-0-80">80</a></span>
<span class="normal"><a href="#__codelineno-0-81">81</a></span>
<span class="normal"><a href="#__codelineno-0-82">82</a></span>
<span class="normal"><a href="#__codelineno-0-83">83</a></span>
<span class="normal"><a href="#__codelineno-0-84">84</a></span>
<span class="normal"><a href="#__codelineno-0-85">85</a></span></pre></div></td><td class="code"><div><pre><span></span><code><a id="__codelineno-0-27" name="__codelineno-0-27"></a><span class="k">def</span> <span class="nf">use_model</span><span class="p">(</span>
<a id="__codelineno-0-28" name="__codelineno-0-28"></a> <span class="n">key</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
<a id="__codelineno-0-29" name="__codelineno-0-29"></a> <span class="o">*</span><span class="p">,</span>
<a id="__codelineno-0-30" name="__codelineno-0-30"></a> <span class="n">version</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="n">Literal</span><span class="p">[</span><span class="s2">&quot;latest&quot;</span><span class="p">]]</span> <span class="o">=</span> <span class="s2">&quot;latest&quot;</span><span class="p">,</span>
<a id="__codelineno-0-31" name="__codelineno-0-31"></a> <span class="n">model_kwarg_name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;model&quot;</span><span class="p">,</span>
<a id="__codelineno-0-32" name="__codelineno-0-32"></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Callable</span><span class="p">[[</span><span class="n">F</span><span class="p">],</span> <span class="n">F</span><span class="p">]:</span>
<a id="__codelineno-0-33" name="__codelineno-0-33"></a> <span class="sd">&quot;&quot;&quot;Inject a model into a function.</span>
<a id="__codelineno-0-34" name="__codelineno-0-34"></a>
<a id="__codelineno-0-35" name="__codelineno-0-35"></a><span class="sd"> Load a model specified by `key` and `version` using the currently active `LargeFile`</span>
<a id="__codelineno-0-36" name="__codelineno-0-36"></a><span class="sd"> implementation. If it&#39;s a single object, it is deserialised using `dill`. If it&#39;s a</span>
<a id="__codelineno-0-37" name="__codelineno-0-37"></a><span class="sd"> directory of files, a `pathlib.Path` instance is given.</span>
<a id="__codelineno-0-38" name="__codelineno-0-38"></a>
<a id="__codelineno-0-39" name="__codelineno-0-39"></a><span class="sd"> By default, the function&#39;s `model` parameter is replaced by the loaded model. This</span>
<a id="__codelineno-0-40" name="__codelineno-0-40"></a><span class="sd"> can be customised by changing `model_kwarg_name`. Multiple models can be loaded by</span>
<a id="__codelineno-0-41" name="__codelineno-0-41"></a><span class="sd"> decorating the same function with `use_model` multiple times.</span>
<a id="__codelineno-0-42" name="__codelineno-0-42"></a>
<a id="__codelineno-0-43" name="__codelineno-0-43"></a><span class="sd"> Examples:</span>
<a id="__codelineno-0-44" name="__codelineno-0-44"></a><span class="sd"> &gt;&gt;&gt; from great_ai import save_model</span>
<a id="__codelineno-0-45" name="__codelineno-0-45"></a><span class="sd"> &gt;&gt;&gt; save_model(3, &#39;my_number&#39;)</span>
<a id="__codelineno-0-46" name="__codelineno-0-46"></a><span class="sd"> &#39;my_number:...&#39;</span>
<a id="__codelineno-0-47" name="__codelineno-0-47"></a><span class="sd"> &gt;&gt;&gt; @use_model(&#39;my_number&#39;)</span>
<a id="__codelineno-0-48" name="__codelineno-0-48"></a><span class="sd"> ... def my_function(a, model):</span>
<a id="__codelineno-0-49" name="__codelineno-0-49"></a><span class="sd"> ... return a + model</span>
<a id="__codelineno-0-50" name="__codelineno-0-50"></a><span class="sd"> &gt;&gt;&gt; my_function(4)</span>
<a id="__codelineno-0-51" name="__codelineno-0-51"></a><span class="sd"> 7</span>
<a id="__codelineno-0-52" name="__codelineno-0-52"></a>
<a id="__codelineno-0-53" name="__codelineno-0-53"></a><span class="sd"> Args:</span>
<a id="__codelineno-0-54" name="__codelineno-0-54"></a><span class="sd"> key: The model&#39;s name as stored by the LargeFile implementation.</span>
<a id="__codelineno-0-55" name="__codelineno-0-55"></a><span class="sd"> version: The model&#39;s version as stored by the LargeFile implementation.</span>
<a id="__codelineno-0-56" name="__codelineno-0-56"></a><span class="sd"> model_kwarg_name: the parameter to use for injecting the loaded model</span>
<a id="__codelineno-0-57" name="__codelineno-0-57"></a><span class="sd"> Returns:</span>
<a id="__codelineno-0-58" name="__codelineno-0-58"></a><span class="sd"> A decorator for model injection.</span>
<a id="__codelineno-0-59" name="__codelineno-0-59"></a><span class="sd"> &quot;&quot;&quot;</span>
<a id="__codelineno-0-60" name="__codelineno-0-60"></a>
<a id="__codelineno-0-61" name="__codelineno-0-61"></a> <span class="k">assert</span> <span class="p">(</span>
<a id="__codelineno-0-62" name="__codelineno-0-62"></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">version</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span> <span class="ow">or</span> <span class="n">version</span> <span class="o">==</span> <span class="s2">&quot;latest&quot;</span>
<a id="__codelineno-0-63" name="__codelineno-0-63"></a> <span class="p">),</span> <span class="s2">&quot;Only integers or the string literal `latest` is allowed as a version&quot;</span>
<span class="normal"><a href="#__codelineno-0-74">74</a></span></pre></div></td><td class="code"><div><pre><span></span><code><a id="__codelineno-0-16" name="__codelineno-0-16"></a><span class="k">def</span> <span class="nf">use_model</span><span class="p">(</span>
<a id="__codelineno-0-17" name="__codelineno-0-17"></a> <span class="n">key</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
<a id="__codelineno-0-18" name="__codelineno-0-18"></a> <span class="o">*</span><span class="p">,</span>
<a id="__codelineno-0-19" name="__codelineno-0-19"></a> <span class="n">version</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="n">Literal</span><span class="p">[</span><span class="s2">&quot;latest&quot;</span><span class="p">]]</span> <span class="o">=</span> <span class="s2">&quot;latest&quot;</span><span class="p">,</span>
<a id="__codelineno-0-20" name="__codelineno-0-20"></a> <span class="n">model_kwarg_name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;model&quot;</span><span class="p">,</span>
<a id="__codelineno-0-21" name="__codelineno-0-21"></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Callable</span><span class="p">[[</span><span class="n">F</span><span class="p">],</span> <span class="n">F</span><span class="p">]:</span>
<a id="__codelineno-0-22" name="__codelineno-0-22"></a> <span class="sd">&quot;&quot;&quot;Inject a model into a function.</span>
<a id="__codelineno-0-23" name="__codelineno-0-23"></a>
<a id="__codelineno-0-24" name="__codelineno-0-24"></a><span class="sd"> Load a model specified by `key` and `version` using the currently active `LargeFile`</span>
<a id="__codelineno-0-25" name="__codelineno-0-25"></a><span class="sd"> implementation. If it&#39;s a single object, it is deserialised using `dill`. If it&#39;s a</span>
<a id="__codelineno-0-26" name="__codelineno-0-26"></a><span class="sd"> directory of files, a `pathlib.Path` instance is given.</span>
<a id="__codelineno-0-27" name="__codelineno-0-27"></a>
<a id="__codelineno-0-28" name="__codelineno-0-28"></a><span class="sd"> By default, the function&#39;s `model` parameter is replaced by the loaded model. This</span>
<a id="__codelineno-0-29" name="__codelineno-0-29"></a><span class="sd"> can be customised by changing `model_kwarg_name`. Multiple models can be loaded by</span>
<a id="__codelineno-0-30" name="__codelineno-0-30"></a><span class="sd"> decorating the same function with `use_model` multiple times.</span>
<a id="__codelineno-0-31" name="__codelineno-0-31"></a>
<a id="__codelineno-0-32" name="__codelineno-0-32"></a><span class="sd"> Examples:</span>
<a id="__codelineno-0-33" name="__codelineno-0-33"></a><span class="sd"> &gt;&gt;&gt; from great_ai import save_model</span>
<a id="__codelineno-0-34" name="__codelineno-0-34"></a><span class="sd"> &gt;&gt;&gt; save_model(3, &#39;my_number&#39;)</span>
<a id="__codelineno-0-35" name="__codelineno-0-35"></a><span class="sd"> &#39;my_number:...&#39;</span>
<a id="__codelineno-0-36" name="__codelineno-0-36"></a><span class="sd"> &gt;&gt;&gt; @use_model(&#39;my_number&#39;)</span>
<a id="__codelineno-0-37" name="__codelineno-0-37"></a><span class="sd"> ... def my_function(a, model):</span>
<a id="__codelineno-0-38" name="__codelineno-0-38"></a><span class="sd"> ... return a + model</span>
<a id="__codelineno-0-39" name="__codelineno-0-39"></a><span class="sd"> &gt;&gt;&gt; my_function(4)</span>
<a id="__codelineno-0-40" name="__codelineno-0-40"></a><span class="sd"> 7</span>
<a id="__codelineno-0-41" name="__codelineno-0-41"></a>
<a id="__codelineno-0-42" name="__codelineno-0-42"></a><span class="sd"> Args:</span>
<a id="__codelineno-0-43" name="__codelineno-0-43"></a><span class="sd"> key: The model&#39;s name as stored by the LargeFile implementation.</span>
<a id="__codelineno-0-44" name="__codelineno-0-44"></a><span class="sd"> version: The model&#39;s version as stored by the LargeFile implementation.</span>
<a id="__codelineno-0-45" name="__codelineno-0-45"></a><span class="sd"> model_kwarg_name: the parameter to use for injecting the loaded model</span>
<a id="__codelineno-0-46" name="__codelineno-0-46"></a><span class="sd"> Returns:</span>
<a id="__codelineno-0-47" name="__codelineno-0-47"></a><span class="sd"> A decorator for model injection.</span>
<a id="__codelineno-0-48" name="__codelineno-0-48"></a><span class="sd"> &quot;&quot;&quot;</span>
<a id="__codelineno-0-49" name="__codelineno-0-49"></a>
<a id="__codelineno-0-50" name="__codelineno-0-50"></a> <span class="k">assert</span> <span class="p">(</span>
<a id="__codelineno-0-51" name="__codelineno-0-51"></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">version</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span> <span class="ow">or</span> <span class="n">version</span> <span class="o">==</span> <span class="s2">&quot;latest&quot;</span>
<a id="__codelineno-0-52" name="__codelineno-0-52"></a> <span class="p">),</span> <span class="s2">&quot;Only integers or the string literal `latest` is allowed as a version&quot;</span>
<a id="__codelineno-0-53" name="__codelineno-0-53"></a>
<a id="__codelineno-0-54" name="__codelineno-0-54"></a> <span class="n">model</span><span class="p">,</span> <span class="n">actual_version</span> <span class="o">=</span> <span class="n">_load_model</span><span class="p">(</span>
<a id="__codelineno-0-55" name="__codelineno-0-55"></a> <span class="n">key</span><span class="o">=</span><span class="n">key</span><span class="p">,</span>
<a id="__codelineno-0-56" name="__codelineno-0-56"></a> <span class="n">version</span><span class="o">=</span><span class="kc">None</span> <span class="k">if</span> <span class="n">version</span> <span class="o">==</span> <span class="s2">&quot;latest&quot;</span> <span class="k">else</span> <span class="n">version</span><span class="p">,</span>
<a id="__codelineno-0-57" name="__codelineno-0-57"></a> <span class="p">)</span>
<a id="__codelineno-0-58" name="__codelineno-0-58"></a>
<a id="__codelineno-0-59" name="__codelineno-0-59"></a> <span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="n">func</span><span class="p">:</span> <span class="n">F</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">F</span><span class="p">:</span>
<a id="__codelineno-0-60" name="__codelineno-0-60"></a> <span class="n">assert_function_is_not_finalised</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<a id="__codelineno-0-61" name="__codelineno-0-61"></a>
<a id="__codelineno-0-62" name="__codelineno-0-62"></a> <span class="n">store</span> <span class="o">=</span> <span class="n">get_function_metadata_store</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<a id="__codelineno-0-63" name="__codelineno-0-63"></a> <span class="n">store</span><span class="o">.</span><span class="n">model_parameter_names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">model_kwarg_name</span><span class="p">)</span>
<a id="__codelineno-0-64" name="__codelineno-0-64"></a>
<a id="__codelineno-0-65" name="__codelineno-0-65"></a> <span class="n">model</span><span class="p">,</span> <span class="n">actual_version</span> <span class="o">=</span> <span class="n">_load_model</span><span class="p">(</span>
<a id="__codelineno-0-66" name="__codelineno-0-66"></a> <span class="n">key</span><span class="o">=</span><span class="n">key</span><span class="p">,</span>
<a id="__codelineno-0-67" name="__codelineno-0-67"></a> <span class="n">version</span><span class="o">=</span><span class="kc">None</span> <span class="k">if</span> <span class="n">version</span> <span class="o">==</span> <span class="s2">&quot;latest&quot;</span> <span class="k">else</span> <span class="n">version</span><span class="p">,</span>
<a id="__codelineno-0-68" name="__codelineno-0-68"></a> <span class="p">)</span>
<a id="__codelineno-0-69" name="__codelineno-0-69"></a>
<a id="__codelineno-0-70" name="__codelineno-0-70"></a> <span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="n">func</span><span class="p">:</span> <span class="n">F</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">F</span><span class="p">:</span>
<a id="__codelineno-0-71" name="__codelineno-0-71"></a> <span class="n">assert_function_is_not_finalised</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<a id="__codelineno-0-72" name="__codelineno-0-72"></a>
<a id="__codelineno-0-73" name="__codelineno-0-73"></a> <span class="n">store</span> <span class="o">=</span> <span class="n">get_function_metadata_store</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<a id="__codelineno-0-74" name="__codelineno-0-74"></a> <span class="n">store</span><span class="o">.</span><span class="n">model_parameter_names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">model_kwarg_name</span><span class="p">)</span>
<a id="__codelineno-0-75" name="__codelineno-0-75"></a>
<a id="__codelineno-0-76" name="__codelineno-0-76"></a> <span class="nd">@wraps</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<a id="__codelineno-0-77" name="__codelineno-0-77"></a> <span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Any</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">Any</span><span class="p">:</span>
<a id="__codelineno-0-78" name="__codelineno-0-78"></a> <span class="n">tracing_context</span> <span class="o">=</span> <span class="n">TracingContext</span><span class="o">.</span><span class="n">get_current_tracing_context</span><span class="p">()</span>
<a id="__codelineno-0-79" name="__codelineno-0-79"></a> <span class="k">if</span> <span class="n">tracing_context</span><span class="p">:</span>
<a id="__codelineno-0-80" name="__codelineno-0-80"></a> <span class="n">tracing_context</span><span class="o">.</span><span class="n">log_model</span><span class="p">(</span><span class="n">Model</span><span class="p">(</span><span class="n">key</span><span class="o">=</span><span class="n">key</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="n">actual_version</span><span class="p">))</span>
<a id="__codelineno-0-81" name="__codelineno-0-81"></a> <span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">,</span> <span class="o">**</span><span class="p">{</span><span class="n">model_kwarg_name</span><span class="p">:</span> <span class="n">model</span><span class="p">})</span>
<a id="__codelineno-0-82" name="__codelineno-0-82"></a>
<a id="__codelineno-0-83" name="__codelineno-0-83"></a> <span class="k">return</span> <span class="n">cast</span><span class="p">(</span><span class="n">F</span><span class="p">,</span> <span class="n">wrapper</span><span class="p">)</span>
<a id="__codelineno-0-84" name="__codelineno-0-84"></a>
<a id="__codelineno-0-85" name="__codelineno-0-85"></a> <span class="k">return</span> <span class="n">decorator</span>
<a id="__codelineno-0-65" name="__codelineno-0-65"></a> <span class="nd">@wraps</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<a id="__codelineno-0-66" name="__codelineno-0-66"></a> <span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Any</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">Any</span><span class="p">:</span>
<a id="__codelineno-0-67" name="__codelineno-0-67"></a> <span class="n">tracing_context</span> <span class="o">=</span> <span class="n">TracingContext</span><span class="o">.</span><span class="n">get_current_tracing_context</span><span class="p">()</span>
<a id="__codelineno-0-68" name="__codelineno-0-68"></a> <span class="k">if</span> <span class="n">tracing_context</span><span class="p">:</span>
<a id="__codelineno-0-69" name="__codelineno-0-69"></a> <span class="n">tracing_context</span><span class="o">.</span><span class="n">log_model</span><span class="p">(</span><span class="n">Model</span><span class="p">(</span><span class="n">key</span><span class="o">=</span><span class="n">key</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="n">actual_version</span><span class="p">))</span>
<a id="__codelineno-0-70" name="__codelineno-0-70"></a> <span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">,</span> <span class="o">**</span><span class="p">{</span><span class="n">model_kwarg_name</span><span class="p">:</span> <span class="n">model</span><span class="p">})</span>
<a id="__codelineno-0-71" name="__codelineno-0-71"></a>
<a id="__codelineno-0-72" name="__codelineno-0-72"></a> <span class="k">return</span> <span class="n">cast</span><span class="p">(</span><span class="n">F</span><span class="p">,</span> <span class="n">wrapper</span><span class="p">)</span>
<a id="__codelineno-0-73" name="__codelineno-0-73"></a>
<a id="__codelineno-0-74" name="__codelineno-0-74"></a> <span class="k">return</span> <span class="n">decorator</span>
</code></pre></div></td></tr></table></div>
</details>
</div>