Deployed 8d1ad9a with MkDocs version: 1.3.0

how-to-guides/use-service/use-service.md

@@ -0,0 +1,131 @@
+# How to use a GreatAI service
+
+After [creating a GreatAI service](/how-to-guides/create-service) by wrapping your prediction function, and optionally [configuring it](/how-to-guides/configure-service), it's time to do some prediction.
+
+Let's take the following example:
+
+```python title="greeter.py"
+from great_ai import GreatAI
+
+@GreatAI.create
+def greeter(your_name: str) -> str:
+    return f'Hi {your_name}'
+```
+
+## One-off prediction
+
+Even though `greeter` is now an instance of [GreatAI][great_ai.GreatAI], you can continue using it as a regular function.
+
+```python
+>>> greeter('Bob')
+Trace[str]({'created': '2022-07-11T14:31:46.183764',
+  'exception': None,
+  'feedback': None,
+  'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
+  'models': [],
+  'original_execution_time_ms': 0.0381,
+  'output': 'Hi Bob',
+  'tags': ['greeter', 'online', 'development'],
+  'trace_id': '7c284fd7-7f0d-4464-b5f8-3ef126df34af'})
+```
+
+As you can see, the original return value is wrapped in a [Trace][great_ai.Trace] object (which is also persisted in your database of choice). You can access the original value under the `output` property.
+
+## Online prediction
+
+Likely, the main way you would like to expose your model is through an HTTP API. [@GreatAI.create][great_ai.GreatAI.create] scaffolds many REST API endpoints for your model and creates a [FastAPI](https://fastapi.tiangolo.com/){ target=_blank } app available under [GreatAI.app][great_ai.GreatAI]. This can be served using [uvicorn](https://www.uvicorn.org/){ target=_blank } or any other [ASGI server](https://asgi.readthedocs.io/en/latest/){ target=_blank }.
+
+Since most ML code lives in [Jupyter](https://jupyter.org/){ target=_blank } notebooks, therefore, deploying a notebook containing the inference function is supported. To this end, `uvicorn` is wrapped by the `great-ai` command-line utility which, among others, takes care of feeding a notebook into `uvicorn`. It also supports auto-reloading.
+
+### In development
+
+```sh
+great-ai greeter.py
+```
+
+!!! success
+    Your model is accessible at [localhost:6060](http:/127.0.0.1:6060){ target=_blank }.
+
+Some configuration options are also supported.
+
+```sh
+great-ai greeter.py --port 8000 --host 127.0.0.1 --timeout_keep_alive 10
+```
+> For more options (but no Notebook support, use [uvicorn](https://www.uvicorn.org/){ target=_blank })
+
+### In production
+
+There are three main approaches for deploying a GreatAI service.
+
+#### Manual deployment
+
+The app is run in *production-mode* if the value of the `ENVIRONMENT` environment variable is set to `production`.
+
+```sh
+ENVIRONMENT=production great-ai greeter.py
+```
+
+Simply run `ENVIRONMENT=production great-ai deploy.ipynb` in the command-line of a production machine.
+> This is the crudest approach, however, it might be fitting for some contexts.
+
+#### Containerised deployment
+
+Run the notebook directly in a container or create a service for it using your favourite container orchestrator.
+
+```sh
+docker run -p 6060:6060 --volume `pwd`:/app --rm \
+  schmelczera/great-ai deploy.ipynb
+```
+> You can replace ``pwd`` with the path to your code's folder.
+
+#### Use a Platform-as-a-Service
+
+Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers (such as [DO App platform](https://www.digitalocean.com/products/app-platform){ target=_blank } or [MLEM](https://mlem.ai/){ target=_blank }) that take a Docker image as a source and handle the rest of the deployment.
+
+To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as [PyTorch](https://pytorch.org/){ target=_blank } or [TensorFlow](https://www.tensorflow.org/){ target=_blank }.
+
+```Dockerfile
+FROM schmelczera/great-ai:latest
+
+# Remove this block if you don't have a requirements.txt
+COPY requirements.txt ./   
+RUN pip install --no-cache-dir --requirement requirements.txt
+
+# If you store your models in S3 or GridFS, it may be a 
+# good idea to cache them in the image so that you don't
+# have to download it each time a container starts
+RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor
+
+# Add you application code to the image
+COPY . .
+
+# The default ENTRYPOINT is great-ai, specify it's argument using CMD
+CMD ["deploy.ipynb"]
+
+```
+
+## Batch prediction
+
+Processing larger amounts of data on a single machine is made easy by the [GreatAI][great_ai.GreatAI]'s [process_batch][great_ai.GreatAI.process_batch] method. This relies on multiprocessing ([parallel_map][great_ai.utilities.parallel_map.parallel_map.parallel_map]) to take full advantage of all available CPU-cores.
+
+```python
+>>> greeter.process_batch(['Alice', 'Bob'])
+[Trace[str]({'created': '2022-07-11T14:36:37.119183',
+   'exception': None,
+   'feedback': None,
+   'logged_values': {'arg:your_name:length': 5, 'arg:your_name:value': 'Alice'},
+   'models': [],
+   'original_execution_time_ms':  0.1251,
+   'output': 'Hi Alice',
+   'tags': ['greeter', 'online', 'development'],
+   'trace_id': '90ffa15f-e839-41c4-8e7a-3211168bc138'}),
+ Trace[str]({'created': '2022-07-11T14:36:37.166659',
+   'exception': None,
+   'feedback': None,
+   'logged_values': {'arg:your_name:length': 3, 'arg:your_name:value': 'Bob'},
+   'models': [],
+   'original_execution_time_ms':  0.0571,
+   'output': 'Hi Bob',
+   'tags': ['greeter', 'online', 'development'],
+   'trace_id': 'f48e94c7-0815-48b3-a864-41349d3dae84'})]
+```