diff --git a/.vscode/settings.json b/.vscode/settings.json index 480d3d9..99a8cc7 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -21,6 +21,7 @@ "initialising", "inplace", "ipynb", + "langcodes", "lemmatize", "levelname", "levelno", @@ -68,12 +69,11 @@ }, "notebook.output.textLineLimit": 400, "python.defaultInterpreterPath": ".env/bin/python", - "python.testing.pytestArgs": [ - "tests" - ], + "python.testing.pytestArgs": ["tests"], + "editor.rulers": [88], "python.linting.flake8Enabled": false, "python.linting.pylintEnabled": false, "python.linting.mypyEnabled": true, "python.testing.unittestEnabled": false, - "python.testing.pytestEnabled": true, + "python.testing.pytestEnabled": true } diff --git a/README.md b/README.md index 6f4a8b0..e273b6e 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,10 @@ > GreatAI helps you easily transform your prototype AI code into production-ready software. +[![Sonar line coverage](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=coverage)](https://sonar.scoutinscience.com/dashboard?id=great-ai) +[![Sonar technical debt](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=sqale_index)](https://sonar.scoutinscience.com/dashboard?id=great-ai) +[![Sonar LoC](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=ncloc)](https://sonar.scoutinscience.com/dashboard?id=great-ai) [![Test](https://github.com/schmelczer/great-ai/actions/workflows/test.yml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/test.yml) -[![Quality Gate Status](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=alert_status)](https://sonar.schmelczer.com/dashboard?id=great-ai) [![Publish on PyPI](https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml) [![Publish on DockerHub](https://github.com/schmelczer/great-ai/actions/workflows/docker.yaml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/docker.yaml) [![Downloads](https://pepy.tech/badge/great-ai/month)](https://pepy.tech/project/great-ai) diff --git a/docs/how-to-guides/index.md b/docs/how-to-guides/call-remote.md similarity index 100% rename from docs/how-to-guides/index.md rename to docs/how-to-guides/call-remote.md diff --git a/docs/how-to-guides/create-service.md b/docs/how-to-guides/create-service.md new file mode 100644 index 0000000..104975d --- /dev/null +++ b/docs/how-to-guides/create-service.md @@ -0,0 +1,118 @@ +# How to instantiate a GreatAI service + +The core value of the `great-ai` library lies in its [great_ai.deploy.GreatAI][] class. In order to take advantage of it, you need to create an instance wrapping your code. + +Let's say that you have the following greeter function: + +```python title="greeter.py" +def my_greeter_function(your_name): + return f'Hi {your_name}!' +``` + +You can simply decorate (wrap) this function with the `GreatAI.create` factory. + +```python title="greeter.py" +from great_ai import GreatAI + +@GreatAI.create +def greeter(your_name): + return f'Hi {your_name}!' +``` + +??? info "Why not simply use `@GreatAI?`" + The purpose of the `GreatAI.create` is simply to provide you with type-checking through MyPy, Pylance, and similar libraries. However, the overloading support for `__new__` is lacking in MyPy, thus, a static factory method is used instead. + +## With types + +[Type annotating your codebase](https://realpython.com/python-type-checking/){ target=_blank } can save you from lots of trivial mistakes, that's why it's highly advised. Simply add the expected types to your function's signature. + +```python title="type_safe_greeter.py" +from great_ai import GreatAI + +@GreatAI.create +def type_safe_greeter(your_name: str) -> str: + return f'Hi {your_name}!' +``` + +This not only allows you to statically typecheck your code, but by default, GreatAI will check it during runtime as well using [typeguard](https://github.com/agronholm/typeguard){ target=_blank }. + +## With async + +Asynchronous code can result in immense performance gains in certain cases. For example, you might rely on a third-party service, do database access, or [call a remote GreatAI instance](/how-to-guides/call-remote). In these cases, you can simply make your function `async` without any other changes. + +```python title="async_greeter.py" +from great_ai import GreatAI +from asyncio import sleep + +@GreatAI.create +async def async_greeter(your_name: str) -> str: + await sleep(2) # simulate IO-heavy operation + return f'Hi {your_name}!' +``` + +## With decorators + +GreatAI can decorate already decorated functions. The only restriction is that `@GreatAI.create` always have to come last. There are two built-in decorators that you can use to customise your function. + +### Using `use_model` + +If you have previously saved a model with `save_model`, you can inject it into your function by calling `use_model`. + +```python title="greeter_with_model.py" +from great_ai import GreatAI, use_model + +@GreatAI.create +@use_model('name_of_my_model', version='latest') #(1) +def type_safe_greeter(your_name: str, model) -> str: + return f'Hi {your_name}!' + +assert type_safe_greeter('Andras').output == 'Hi Andras' +``` + +1. By default, the parameter named `model` will be replaced by the loaded model. This behaviour can be customised by setting the `model_kwarg_name`. This way, even multiple models can be injected into a single function. + +!!! important + You must call `@use_model` before `GreatAI.create`. Feel free to use `@use_model` in other places of the code base, it works equally well outside of GreatAI services. + + +### Using `parameter` + +If you wish to turn of logging or specify custom validation for your parameters, you can use the `@parameter` decorator. + +!!! note + By default, all parameters that are not affected by an explicit `@parameter` or `@use_model` decorator, are automatically decorated with `@parameter` when `GreatAI.create` is called. + +```python "greeter_with_validation.py" +from great_ai import GreatAI, use_model + +@GreatAI.create +@use_model('name_of_my_model', version='latest') +def type_safe_greeter(your_name: str, model) -> str: + return f'Hi {your_name}!' + +assert type_safe_greeter('Andras').output == 'Hi Andras' +``` + +!!! important + You must call `@parameter` before `GreatAI.create`. Feel free to use `@parameter` in other places of the code base, it works equally well outside of GreatAI services. + + +## Complex example + +Refer to the following example summarising the options you have when instantiating a GreatAI service. + +```python title="complex.py" +from great_ai import save_model, GreatAI, parameter, use_model + +save_model(4, 'secret-number') #(2) + +@GreatAI.create +@parameter('positive_number', validator=lambda n: n > 0) +@use_model('secret-number', version='latest', model_kwarg_name='secret') +def add_number(positive_number: int, secret: int) -> int: + return positive_number + secret + +assert add_number(1).output == 5 +``` + +2. Refer to [storing models](/how-to-guides/store-models) for specifying where to store your models. diff --git a/docs/how-to-guides/store-models.md b/docs/how-to-guides/store-models.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/how-to-guides/using-service.md b/docs/how-to-guides/using-service.md new file mode 100644 index 0000000..6ed65bb --- /dev/null +++ b/docs/how-to-guides/using-service.md @@ -0,0 +1,14 @@ +# How to use a GreatAI service + +After [creating a GreatAI service](/how-to-guides/cerate-service) by wrapping your prediction function, it's time to do some prediction. + +Let's use the following example: + +```python "type_safe_greeter.py" +from great_ai import GreatAI + +@GreatAI.create +def type_safe_greeter(your_name: str) -> str: + return f'Hi {your_name}' +``` + diff --git a/docs/index.md b/docs/index.md index d24e689..9db1204 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,13 +3,15 @@ -[![Test](https://github.com/schmelczer/great-ai/actions/workflows/test.yml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/check.yml) -[![Quality Gate Status](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=alert_status)](https://sonar.schmelczer.com/dashboard?id=great-ai) +[![Sonar line coverage](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=coverage)](https://sonar.scoutinscience.com/dashboard?id=great-ai) +[![Sonar technical debt](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=sqale_index)](https://sonar.scoutinscience.com/dashboard?id=great-ai) +[![Sonar LoC](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=ncloc)](https://sonar.scoutinscience.com/dashboard?id=great-ai) +[![Test](https://github.com/schmelczer/great-ai/actions/workflows/test.yml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/test.yml) [![Publish on PyPI](https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml) [![Publish on DockerHub](https://github.com/schmelczer/great-ai/actions/workflows/docker.yaml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/docker.yaml) [![Downloads](https://pepy.tech/badge/great-ai/month)](https://pepy.tech/project/great-ai) -Applying AI is becoming increasingly easier but many case studies have shown that these applications are often deployed poorly. This may lead to suboptimal performance and to introducing [unintended biases](https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction){ target=_blank }. To extend the list of available solutions, ==GreatAI helps you easily transform your prototype AI code into production-ready software.== +Applying AI is becoming increasingly easier but many case studies have shown that these applications are often deployed poorly. This may lead to suboptimal performance and to introducing [unintended biases](https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction){ target=_blank }. GreatAI helps fixing this by allowing you to ==easily transform your prototype AI code into production-ready software==. ??? quote "Case studies" "There is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature." @@ -36,9 +38,10 @@ Applying AI is becoming increasingly easier but many case studies have shown tha - [x] A simple, unified configuration interface - [x] Fully-typed API for [Pylance](https://github.com/microsoft/pylance-release){ target=_blank } and [MyPy](http://mypy-lang.org){ target=_blank } support - [x] Auto-reload for development -- [x] Deployable Jupyter Notebooks - [x] Docker support for deployment -- [x] Dashboard for high-level overview and searching traces +- [x] Deployable Jupyter Notebooks +- [x] Dashboard for high-level overview and analysing traces +- [ ] Support for direct file input ## Hello world @@ -59,7 +62,7 @@ def hello_world(name: str) -> str: #(2) 2. it gets a `process_batch` method for supporting parallel execution, 3. and it can be deployed using the `great-ai` command-line tool. -2. [Typing functions](https://docs.python.org/3/library/typing.html){ target=_blank } is recommended in general, however, not necessary for GreatAI to work. +2. [Typing functions](https://docs.python.org/3/library/typing.html){ target=_blank } is recommended in general, however, not required for GreatAI to work. ??? note In practice, `hello_world` could be an inference function of some AI/ML application. But it could also just wrap a black-box solution of some SaaS. Either ways, it is imperative to have continuos oversight of the services you provide and data you process. @@ -94,7 +97,7 @@ GreatAI fits between the prototype and deployment phase of your (or your organis There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great. -However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many [best-practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort. +However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best-practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
diff --git a/docs/reference.md b/docs/reference.md deleted file mode 100644 index 168b950..0000000 --- a/docs/reference.md +++ /dev/null @@ -1 +0,0 @@ -::: great_ai.utilities.unique \ No newline at end of file diff --git a/docs/reference/index.md b/docs/reference/index.md new file mode 100644 index 0000000..a5a7815 --- /dev/null +++ b/docs/reference/index.md @@ -0,0 +1,56 @@ +# Reference + +```python +from great_ai import * +``` + +## Core + +::: great_ai.GreatAI + options: + filters: ['!__init__'] + show_root_heading: true + +::: great_ai.configure + options: + show_root_heading: true + +::: great_ai.save_model + options: + show_root_heading: true + +::: great_ai.use_model + options: + show_root_heading: true + +::: great_ai.parameter + options: + show_root_heading: true + +::: great_ai.log_metric + options: + show_root_heading: true + +## Remote calls + +::: great_ai.call_remote_great_ai + options: + show_root_heading: true + +::: great_ai.call_remote_great_ai_async + options: + show_root_heading: true + +## Ground-truth + +::: great_ai.add_ground_truth + options: + show_root_heading: true + +::: great_ai.query_ground_truth + options: + show_root_heading: true + +::: great_ai.delete_ground_truth + options: + show_root_heading: true diff --git a/docs/reference/large-file.md b/docs/reference/large-file.md new file mode 100644 index 0000000..19c3b38 --- /dev/null +++ b/docs/reference/large-file.md @@ -0,0 +1,5 @@ +# LargeFile + +```python +from great_ai.large_file import * +``` diff --git a/docs/reference/utilities.md b/docs/reference/utilities.md new file mode 100644 index 0000000..24a9b06 --- /dev/null +++ b/docs/reference/utilities.md @@ -0,0 +1,38 @@ +# Utilities + +```python +from great_ai.utilities import * +``` + +## NLP tools + +Well-tested tools that can be used in production with confidence. The toolbox of feature-extraction functions is expected to grow to cover other domains as well. + +::: great_ai.utilities.clean +::: great_ai.utilities.get_sentences +::: great_ai.utilities.language.predict_language +::: great_ai.utilities.language.is_english +::: great_ai.utilities.language.english_name_of_language +::: great_ai.utilities.evaluate_ranking.evaluate_ranking + +## Parallel processing + +Multiprocessing and multithreading-based parallelism with support for `async` functions. Its main purpose is to implement [great_ai.GreatAI.process_batch][], however, the parallel processing functions are also convenient for covering other types of mapping needs with a friendlier API than [joblib](https://joblib.readthedocs.io/en/latest/parallel.html){ target=_blank } or [multiprocess](https://pypi.org/project/multiprocess/){ target=_blank }. + +::: great_ai.utilities.parallel_map.simple_parallel_map +::: great_ai.utilities.parallel_map.parallel_map +::: great_ai.utilities.parallel_map.threaded_parallel_map + +## Composable parallel processing + +Because both [threaded_parallel_map][great_ai.utilities.parallel_map.threaded_parallel_map.threaded_parallel_map] and [parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map] have a streaming interface, it is easy to compose them and end up with, for example, a process for each CPU core with its own thread-pool or event-loop. Longer pipelines are also easy to imagine. The chunking methods help in these compositions. + +For more info, check-out [the scraping guide](/how-to-guides/scraping). + +::: great_ai.utilities.chunk +::: great_ai.utilities.unchunk + +## Operations + +::: great_ai.utilities.config_file.config_file +::: great_ai.utilities.logger.get_logger diff --git a/docs/reference/views.md b/docs/reference/views.md new file mode 100644 index 0000000..7b85ebe --- /dev/null +++ b/docs/reference/views.md @@ -0,0 +1,3 @@ +# View models + +::: great_ai.views.trace.Trace diff --git a/docs/tutorial/index.md b/docs/tutorial/index.md index 8417e5d..b32f7ac 100644 --- a/docs/tutorial/index.md +++ b/docs/tutorial/index.md @@ -1,16 +1,19 @@ # Train and deploy a SOTA model +Let's see GreatAI in action by going over the life-cycle of a simple service. + +## Objectives + +1. You will see how the [great_ai.utilities][] can integrate into your Data Science workflow. +2. You will use [great_ai.large_file][] to version and store your trained model. +3. You will use [GreatAI][great_ai.GreatAI] to prepare your model for a robust and responsible deployment. + ## Overview You are going to train a field of study (domain) classifier for scientific sentences. The exact task was proposed by the [SciBERT paper](https://arxiv.org/abs/1903.10676) in which SciBERT [achieved an F1-score of 0.6571](https://paperswithcode.com/sota/sentence-classification-on-paper-field). We are going to outperform it using a trivial text classification model: a [Linear SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html). We use the same synthetic dataset derived from the [Microsoft Academic Graph](https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/). The dataset is [available here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag). -## Objectives - -1. You will see how the `great_ai.utilities` can integrate into your Data Science workflow. -2. You will use `great_ai.large_file` to version and store your trained model -3. You will use `GreatAI` to prepare your model for a robust and responsible deployment !!! success You are ready to start the tutorial. Feel free to come back to the [summary](#summary) section once you're finished. @@ -18,7 +21,7 @@ We use the same synthetic dataset derived from the [Microsoft Academic Graph](ht
[:fontawesome-solid-chart-simple: Train it](train.ipynb){ .md-button .md-button--primary } -[:material-cloud-tags: Deploy it](deploy.ipynb){ .md-button .md-button--secondary } +[:material-cloud-tags: Deploy it](deploy.ipynb){ .md-button .md-button--primary }
@@ -26,11 +29,11 @@ We use the same synthetic dataset derived from the [Microsoft Academic Graph](ht ### The [training notebook](train.ipynb) -We load and preprocess the dataset while relying on `great_ai.utilities.clean` for the heavy-lifting. Additionally, the preprocessing is parallelised using `great_ai.utilities.simple_parallel_map`. +We load and preprocess the dataset while relying on [great_ai.utilities.clean][] for the heavy-lifting. Additionally, the preprocessing is parallelised using [great_ai.utilities.simple_parallel_map][] -After training and evaluating a model, it is exported using `great_ai.save_model`. +After training and evaluating a model, it is exported using [great_ai.save_model][]. -??? tip "Using the cloud" +??? tip "Remote storage" To store your model remotely, you need to set your credentials before calling `save_model`. For example, to use [AWS S3](https://aws.amazon.com/s3/): @@ -51,7 +54,7 @@ After training and evaluating a model, it is exported using `great_ai.save_model ### The [deployment notebook](deploy.ipynb) -We create an inference function that can be hardened by wrapping it in a `great_ai.GreatAI` instance. +We create an inference function that can be hardened by wrapping it in a [GreatAI][great_ai.GreatAI] instance. ```python from great_ai import GreatAI, use_model @@ -64,7 +67,7 @@ def predict_domain(sentence, model): return str(model.predict(inputs)[0]) ``` -1. `@use_model` loads and injects your model into the `predict_domain` function's `model` argument. +1. [@use_model][great_ai.use_model] loads and injects your model into the `predict_domain` function's `model` argument. You can freely reference it knowing that it is always given to the function. Finally, we deploy the model, inference function, and the GreatAI wrapping all of these. For that we either use: `great-ai deploy.ipynb` or build a Docker image. diff --git a/mkdocs.yaml b/mkdocs.yaml index 1f730a2..fc3e149 100644 --- a/mkdocs.yaml +++ b/mkdocs.yaml @@ -6,6 +6,24 @@ repo_url: https://github.com/schmelczer/great-ai repo_name: schmelczer/great-ai edit_uri: edit/main/docs/ +copyright: GNU General Public License v3 + +extra: + generator: false + social: + - icon: fontawesome/solid/code + link: https://schmelczer.dev + name: about the author + - icon: fontawesome/brands/github + link: https://github.com/schmelczer/great-ai + name: great-ai on Github + - icon: fontawesome/brands/docker + link: https://hub.docker.com/repository/docker/schmelczera/great-ai + name: great-ai on DockerHub + - icon: fontawesome/brands/python + link: https://pypi.org/project/great-ai + name: great-ai on PyPI + theme: name: "material" custom_dir: docs/overrides @@ -16,13 +34,17 @@ theme: - content.tooltips palette: - scheme: default + primary: light blue + media: "(prefers-color-scheme: light)" toggle: - icon: material/brightness-7 + icon: material/lightbulb name: Switch to dark mode - scheme: slate + primary: light blue + media: "(prefers-color-scheme: dark)" toggle: - icon: material/brightness-4 + icon: material/lightbulb-outline name: Switch to light mode watch: @@ -31,7 +53,13 @@ watch: plugins: - git-revision-date-localized: - - mkdocstrings + - mkdocstrings: + handlers: + python: + options: + show_root_toc_entry: false + show_root_full_path: false + heading_level: 3 - search - mkdocs-jupyter: include_source: true @@ -61,14 +89,15 @@ markdown_extensions: nav: - Overview: index.md - Tutorial: - - tutorial/index.md + - Tutorial overview: tutorial/index.md - tutorial/train.ipynb - tutorial/deploy.ipynb - User Guides: - - Simple AI lifecycle with classification: - - "Step 1: data": simple/data.ipynb - - "Step 2: train": simple/train.ipynb - - "Step 3: deploy": simple/deploy.ipynb - - reference.md + - how-to-guides/create-service.md + - Reference: + - reference/index.md + - reference/utilities.md + - reference/large-file.md + - reference/views.md - explanation.md