Fix issues from @angyaloliver's feedback

This commit is contained in:
Andras Schmelczer 2022-07-13 20:23:56 +02:00
parent 584d746447
commit 041eefc401
No known key found for this signature in database
GPG key ID: 0EA1BC97D0AB076E
12 changed files with 30 additions and 26 deletions

View file

@ -28,7 +28,7 @@ def greeter(name: str) -> str:
Start it by executing `great-ai demo.py`, find the dashboard at [http://localhost:6060](http://localhost:6060/dashboard). Start it by executing `great-ai demo.py`, find the dashboard at [http://localhost:6060](http://localhost:6060/dashboard).
![demo screen capture](docs/media/demo.gif) ![demo screen capture](https://github.com/schmelczer/great-ai/blob/main/docs/media/demo.gif)
That's it. Your GreatAI service is *nearly* ready for production use. Many of the [SE4ML best-practices](https://se-ml.github.io) are configured and implemented automatically (of course, these can be customised as well). That's it. Your GreatAI service is *nearly* ready for production use. Many of the [SE4ML best-practices](https://se-ml.github.io) are configured and implemented automatically (of course, these can be customised as well).
@ -48,13 +48,13 @@ GreatAI fits between the prototype and deployment phases of your AI development
## Why GreatAI? ## Why GreatAI?
There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker) and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core) provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great. There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker) and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core) provide the most comprehensive suite of features. If you have the opportunity to use them, do that because they're great.
However, [research indicates](https://great-ai.scoutinscience.com) that professionals rarely use them. This may be due to their inherent setup and operating complexity. **GreatAI is designed to be as simple to use as possible.** Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best-practices](https://se-ml.github.io), and thus, can meaningfully improve your deployment without requiring prohibitively large effort. However, [research indicates](https://great-ai.scoutinscience.com) that professionals rarely use them. This may be due to their inherent setup and operating complexity. **GreatAI is designed to be as simple to use as possible.** Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best-practices](https://se-ml.github.io), and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
## Learn more ## Find `great-ai` on [PyPI](https://hub.docker.com/repository/docker/schmelczera/great-ai)
[Check out the documentation](https://great-ai.scoutinscience.com). ## [Learn more](https://great-ai.scoutinscience.com)
## Find `great-ai` on [DockerHub](https://hub.docker.com/repository/docker/schmelczera/great-ai) ## Find `great-ai` on [DockerHub](https://hub.docker.com/repository/docker/schmelczera/great-ai)

View file

@ -3,5 +3,5 @@
This example shows how `great-ai` is used in practice at ScoutinScience. This example shows how `great-ai` is used in practice at ScoutinScience.
<div style="display: flex; justify-content: center;" markdown> <div style="display: flex; justify-content: center;" markdown>
[:material-test-tube: Check out the code](/docs/examples/scibert){ .md-button .md-button--primary } [:material-test-tube: Check out the code](https://github.com/schmelczer/great-ai/tree/main/docs/examples/scibert){ .md-button .md-button--primary }
</div> </div>

View file

@ -62,7 +62,7 @@
"\n", "\n",
"@GreatAI.create\n", "@GreatAI.create\n",
"@use_model(\"small-domain-prediction\", version=\"latest\")\n", "@use_model(\"small-domain-prediction\", version=\"latest\")\n",
"@parameter(\"target_confidence\", validator=lambda c: 0 <= c <= 100)\n", "@parameter(\"target_confidence\", validate=lambda c: 0 <= c <= 100)\n",
"def predict_domain(\n", "def predict_domain(\n",
" text: str, model: Pipeline, target_confidence: int = 50\n", " text: str, model: Pipeline, target_confidence: int = 50\n",
") -> MultiLabelClassificationOutput:\n", ") -> MultiLabelClassificationOutput:\n",

View file

@ -43,6 +43,7 @@ results = [
print(results) print(results)
``` ```
1. Only return the outputs so we don't clutter up the terminal. 1. Only return the outputs so we don't clutter up the terminal.
> Run this script as a regular Python script by executing `python3 client.py`. > Run this script as a regular Python script by executing `python3 client.py`.

View file

@ -76,7 +76,7 @@ assert type_safe_greeter('Andras').output == 'Hi Andras'
### Using `@parameter` ### Using `@parameter`
If you wish to turn of logging or specify custom validation for your parameters, you can use the [@parameter][great_ai.parameter] decorator. If you wish to turn off logging or specify custom validation for your parameters, you can use the [@parameter][great_ai.parameter] decorator.
!!! note !!! note
By default, all parameters that are not affected by an explicit [@parameter][great_ai.parameter] or [@use_model][great_ai.use_model] are automatically decorated with [@parameter][great_ai.parameter] when [@GreatAI.create][great_ai.GreatAI.create] is called. By default, all parameters that are not affected by an explicit [@parameter][great_ai.parameter] or [@use_model][great_ai.use_model] are automatically decorated with [@parameter][great_ai.parameter] when [@GreatAI.create][great_ai.GreatAI.create] is called.
@ -85,7 +85,7 @@ If you wish to turn of logging or specify custom validation for your parameters,
from great_ai import GreatAI, use_model from great_ai import GreatAI, use_model
@GreatAI.create @GreatAI.create
@use_model('name_of_my_model', version='latest') @parameter('your_name', disable_logging=True)
def type_safe_greeter(your_name: str, model) -> str: def type_safe_greeter(your_name: str, model) -> str:
return f'Hi {your_name}!' return f'Hi {your_name}!'
@ -105,7 +105,7 @@ from great_ai import save_model, GreatAI, parameter, use_model, log_metric
save_model(4, 'secret-number') #(1) save_model(4, 'secret-number') #(1)
@GreatAI.create @GreatAI.create
@parameter('positive_number', validator=lambda n: n > 0, disable_logging=True) @parameter('positive_number', validate=lambda n: n > 0, disable_logging=True)
@use_model('secret-number', version='latest', model_kwarg_name='secret') @use_model('secret-number', version='latest', model_kwarg_name='secret')
def add_number(positive_number: int, secret: int) -> int: def add_number(positive_number: int, secret: int) -> int:
log_metric( log_metric(

View file

@ -1,4 +1,4 @@
# How to deploy a GreatAI service # How to perform prediction with GreatAI
After [creating a GreatAI service](/how-to-guides/create-service) by wrapping your prediction function, and optionally [configuring it](/how-to-guides/configure-service), it's time to do some prediction. After [creating a GreatAI service](/how-to-guides/create-service) by wrapping your prediction function, and optionally [configuring it](/how-to-guides/configure-service), it's time to do some prediction.

View file

@ -25,12 +25,12 @@ Applying AI is becoming increasingly easier but many case studies have shown tha
## Features ## Features
- [x] Save prediction traces of each prediction including arguments and model versions - [x] Save prediction traces of each prediction including arguments and model versions
- [x] Save feedback and merge it into a ground-truth database *:arrow_right: quasi-shadow deployment* - [x] Save feedback and merge it into a ground-truth database
- [x] Version and store models and data on shared infrastructure *(MongoDB GridFS, S3-compatible storage, shared local-volume)* - [x] Version and store models and data on shared infrastructure *(MongoDB GridFS, S3-compatible storage, shared local-volume)*
- [x] Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration - [x] Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration
- [x] Input validation - [x] Input validation
- [x] Sensible cache-policy - [x] Sensible cache-policy
- [x] Seamless support for both synchronous and `async` inference methods - [x] Seamless support for both synchronous and asynchronous inference methods
- [x] Easy integration with remote GreatAI instances - [x] Easy integration with remote GreatAI instances
- [x] Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing - [x] Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing
- [x] Well-tested utilities for common NLP tasks (cleaning, language-tagging, sentence-segmentation, etc.) - [x] Well-tested utilities for common NLP tasks (cleaning, language-tagging, sentence-segmentation, etc.)
@ -70,7 +70,7 @@ def greeter(name: str) -> str: #(2)
2. [Typing functions](https://docs.python.org/3/library/typing.html){ target=_blank } is recommended in general, however, not required for GreatAI to work. 2. [Typing functions](https://docs.python.org/3/library/typing.html){ target=_blank } is recommended in general, however, not required for GreatAI to work.
??? note ??? note
In practice, `greeter` could be an inference function of some AI/ML application. But it could also just wrap a black-box solution of some SaaS. Either ways, it is [imperative to have continuos oversight](https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai){ target=_blank } of the services you provide and data you process especially in the context of AI/ML applications. In practice, `greeter` could be an inference function of some AI/ML application. But it could also just wrap a black-box solution of some SaaS. Either way, it is [imperative to have continuous oversight](https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai){ target=_blank } of the services you provide and data you process especially in the context of AI/ML applications.
```sh title="terminal" ```sh title="terminal"
great-ai demo.py great-ai demo.py
@ -96,7 +96,7 @@ GreatAI fits between the prototype and deployment phases of your (or your organi
## Why GreatAI? ## Why GreatAI?
There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great. There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity to use them, do that because they're great.
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. ==GreatAI is designed to be as simple to use as possible.== Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best-practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort. However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. ==GreatAI is designed to be as simple to use as possible.== Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many of the [SE4ML best-practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.

View file

@ -139,7 +139,7 @@
"source": [ "source": [
"Congrats, you've just created your first GreatAI service! 🎉\n", "Congrats, you've just created your first GreatAI service! 🎉\n",
"\n", "\n",
"Now that you've made sure your application is hardened enough for the intended use case, it is time to deploy it. The responsibilities of GreatAI end when it wraps your inference function and model into a production-ready service. You're given the freedom and responsibility to deploy this service. Fortunately, you (or your organisation) probably already has an established routine for deploying services.\n", "Now that you've made sure your application is hardened enough for the intended use case, it is time to deploy it. The responsibilities of GreatAI end when it wraps your inference function and model into a production-ready service. You're given the freedom and responsibility to deploy this service. Fortunately, you (or your organisation) probably already have an established routine for deploying services.\n",
"\n", "\n",
"There are three main approaches to deploy a GreatAI service: For more info about them, check out [the deployment how-to](/how-to-guides/use-service).\n", "There are three main approaches to deploy a GreatAI service: For more info about them, check out [the deployment how-to](/how-to-guides/use-service).\n",
"\n", "\n",

View file

@ -60,7 +60,7 @@
"source": [ "source": [
"Now, we can load the dataset and extract the *training* samples from it. Since we're impatient, we can do it in parallel using the [`simple_parallel_map`](/reference/utilities/#great_ai.utilities.simple_parallel_map) function.\n", "Now, we can load the dataset and extract the *training* samples from it. Since we're impatient, we can do it in parallel using the [`simple_parallel_map`](/reference/utilities/#great_ai.utilities.simple_parallel_map) function.\n",
"\n", "\n",
"> Open files in Python are iterable, with a line being return (in text mode) with each iteration." "> Open files in Python are iterable: in text mode, each iteration returns the next line."
] ]
}, },
{ {

View file

@ -14,7 +14,7 @@ F = TypeVar("F", bound=Callable)
def parameter( def parameter(
parameter_name: str, parameter_name: str,
*, *,
validator: Callable[[Any], bool] = lambda _: True, validate: Callable[[Any], bool] = lambda _: True,
disable_logging: bool = False, disable_logging: bool = False,
) -> Callable[[F], F]: ) -> Callable[[F], F]:
"""Control the validation and logging of function parameters. """Control the validation and logging of function parameters.
@ -30,7 +30,7 @@ def parameter(
... ...
TypeError: type of a must be int; got str instead TypeError: type of a must be int; got str instead
>>> @parameter('positive_a', validator=lambda v: v > 0) >>> @parameter('positive_a', validate=lambda v: v > 0)
... def my_function(positive_a: int): ... def my_function(positive_a: int):
... return a + 2 ... return a + 2
>>> my_function(-1) >>> my_function(-1)
@ -40,7 +40,8 @@ def parameter(
Args: Args:
parameter_name: Name of parameter to consider parameter_name: Name of parameter to consider
validator: Optional validator to run against the concrete argument. ArgumentValidationError is thrown when the return value is False. validate: Optional validate to run against the concrete argument.
ArgumentValidationError is thrown when the return value is False.
disable_logging: Do not save the value in any active TracingContext. disable_logging: Do not save the value in any active TracingContext.
Returns: Returns:
A decorator for argument validation. A decorator for argument validation.
@ -62,9 +63,11 @@ def parameter(
if expected_type is not None: if expected_type is not None:
check_type(parameter_name, argument, expected_type) check_type(parameter_name, argument, expected_type)
if not validator(argument): if not validate(argument):
raise ArgumentValidationError( raise ArgumentValidationError(
f"Argument {parameter_name} in {func.__name__} did not pass validation" f"""Argument {parameter_name} in {
func.__name__
} did not pass validation"""
) )
context = TracingContext.get_current_tracing_context() context = TracingContext.get_current_tracing_context()

View file

@ -24,7 +24,7 @@ async def test_create_trivial_cases() -> None:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_with_parameter() -> None: async def test_with_parameter() -> None:
@GreatAI.create @GreatAI.create
@parameter("name", validator=lambda v: len(v) > 5) @parameter("name", validate=lambda v: len(v) > 5)
async def hello_world(name: str) -> str: async def hello_world(name: str) -> str:
await sleep(0.5) await sleep(0.5)
return f"Hello {name}!" return f"Hello {name}!"
@ -33,7 +33,7 @@ async def test_with_parameter() -> None:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_with_parameters() -> None: async def test_with_parameters() -> None:
@GreatAI.create @GreatAI.create
@parameter("name", validator=lambda v: len(v) > 5) @parameter("name", validate=lambda v: len(v) > 5)
@parameter("unused", disable_logging=True) @parameter("unused", disable_logging=True)
async def hello_world(name: str, unused) -> str: # type: ignore async def hello_world(name: str, unused) -> str: # type: ignore
await sleep(0.5) await sleep(0.5)
@ -45,7 +45,7 @@ async def test_with_parameters() -> None:
def test_wrong_order() -> None: def test_wrong_order() -> None:
with pytest.raises(WrongDecoratorOrderError): with pytest.raises(WrongDecoratorOrderError):
@parameter("name", validator=lambda v: len(v) > 5) @parameter("name", validate=lambda v: len(v) > 5)
@GreatAI.create @GreatAI.create
async def hello_world(name: str) -> str: async def hello_world(name: str) -> str:
return f"Hello {name}!" return f"Hello {name}!"

View file

@ -41,7 +41,7 @@ def test_create_with_other_decorator() -> None:
def test_with_parameter() -> None: def test_with_parameter() -> None:
@GreatAI.create @GreatAI.create
@parameter("name", validator=lambda v: len(v) > 5) @parameter("name", validate=lambda v: len(v) > 5)
def hello_world(name: str) -> str: def hello_world(name: str) -> str:
return f"Hello {name}!" return f"Hello {name}!"
@ -54,7 +54,7 @@ def test_with_parameter() -> None:
def test_wrong_order() -> None: def test_wrong_order() -> None:
with pytest.raises(WrongDecoratorOrderError): with pytest.raises(WrongDecoratorOrderError):
@parameter("name", validator=lambda v: len(v) > 5) @parameter("name", validate=lambda v: len(v) > 5)
@GreatAI.create @GreatAI.create
def hello_world(name: str) -> str: def hello_world(name: str) -> str:
return f"Hello {name}!" return f"Hello {name}!"