Improve docs

This commit is contained in:
Andras Schmelczer 2022-07-10 19:43:39 +02:00
parent c974409fce
commit b97b20ba88
No known key found for this signature in database
GPG key ID: 0EA1BC97D0AB076E
14 changed files with 303 additions and 33 deletions

56
docs/reference/index.md Normal file
View file

@ -0,0 +1,56 @@
# Reference
```python
from great_ai import *
```
## Core
::: great_ai.GreatAI
options:
filters: ['!__init__']
show_root_heading: true
::: great_ai.configure
options:
show_root_heading: true
::: great_ai.save_model
options:
show_root_heading: true
::: great_ai.use_model
options:
show_root_heading: true
::: great_ai.parameter
options:
show_root_heading: true
::: great_ai.log_metric
options:
show_root_heading: true
## Remote calls
::: great_ai.call_remote_great_ai
options:
show_root_heading: true
::: great_ai.call_remote_great_ai_async
options:
show_root_heading: true
## Ground-truth
::: great_ai.add_ground_truth
options:
show_root_heading: true
::: great_ai.query_ground_truth
options:
show_root_heading: true
::: great_ai.delete_ground_truth
options:
show_root_heading: true

View file

@ -0,0 +1,5 @@
# LargeFile
```python
from great_ai.large_file import *
```

View file

@ -0,0 +1,38 @@
# Utilities
```python
from great_ai.utilities import *
```
## NLP tools
Well-tested tools that can be used in production with confidence. The toolbox of feature-extraction functions is expected to grow to cover other domains as well.
::: great_ai.utilities.clean
::: great_ai.utilities.get_sentences
::: great_ai.utilities.language.predict_language
::: great_ai.utilities.language.is_english
::: great_ai.utilities.language.english_name_of_language
::: great_ai.utilities.evaluate_ranking.evaluate_ranking
## Parallel processing
Multiprocessing and multithreading-based parallelism with support for `async` functions. Its main purpose is to implement [great_ai.GreatAI.process_batch][], however, the parallel processing functions are also convenient for covering other types of mapping needs with a friendlier API than [joblib](https://joblib.readthedocs.io/en/latest/parallel.html){ target=_blank } or [multiprocess](https://pypi.org/project/multiprocess/){ target=_blank }.
::: great_ai.utilities.parallel_map.simple_parallel_map
::: great_ai.utilities.parallel_map.parallel_map
::: great_ai.utilities.parallel_map.threaded_parallel_map
## Composable parallel processing
Because both [threaded_parallel_map][great_ai.utilities.parallel_map.threaded_parallel_map.threaded_parallel_map] and [parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map] have a streaming interface, it is easy to compose them and end up with, for example, a process for each CPU core with its own thread-pool or event-loop. Longer pipelines are also easy to imagine. The chunking methods help in these compositions.
For more info, check-out [the scraping guide](/how-to-guides/scraping).
::: great_ai.utilities.chunk
::: great_ai.utilities.unchunk
## Operations
::: great_ai.utilities.config_file.config_file
::: great_ai.utilities.logger.get_logger

3
docs/reference/views.md Normal file
View file

@ -0,0 +1,3 @@
# View models
::: great_ai.views.trace.Trace