Harden and deploy your app¶

Finally, it's time to deploy your model. But before you have to make sure you follow AI deployment best-practices. In the past, this step was too often either the source of unexpected struggles, or worse, simply ignored.

With GreatAI, it has become a matter of 2 lines of code.

In [1]:

from great_ai import GreatAI, use_model
from great_ai.utilities import clean


@GreatAI.create
@use_model("my-domain-predictor")
def predict_domain(sentence, model):
    inputs = [clean(sentence)]
    return str(model.predict(inputs)[0])

from great_ai import GreatAI, use_model from great_ai.utilities import clean @GreatAI.create @use_model("my-domain-predictor") def predict_domain(sentence, model): inputs = [clean(sentence)] return str(model.predict(inputs)[0])

2022-07-09 22:38:56 |  WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️
2022-07-09 22:38:56 |  WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver
2022-07-09 22:38:56 |  WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production
2022-07-09 22:38:56 |  WARNING | Cannot find credentials files, defaulting to using LargeFileLocal
2022-07-09 22:38:56 |     INFO | GreatAI (v0.1.0): configured ✅
2022-07-09 22:38:56 |     INFO |   🔩 tracing_database: ParallelTinyDbDriver
2022-07-09 22:38:56 |     INFO |   🔩 large_file_implementation: LargeFileLocal
2022-07-09 22:38:56 |     INFO |   🔩 is_production: False
2022-07-09 22:38:56 |     INFO |   🔩 should_log_exception_stack: True
2022-07-09 22:38:56 |     INFO |   🔩 prediction_cache_size: 512
2022-07-09 22:38:56 |     INFO |   🔩 dashboard_table_size: 20
2022-07-09 22:38:56 |  WARNING | You still need to check whether you follow all best practices before trusting your deployment.
2022-07-09 22:38:56 |  WARNING | > Find out more at https://se-ml.github.io/practices
2022-07-09 22:38:56 |     INFO | Fetching cached versions of my-domain-predictor
2022-07-09 22:38:56 |     INFO | Latest version of my-domain-predictor is 5 (from versions: 0, 1, 2, 3, 4, 5)
2022-07-09 22:38:56 |     INFO | File my-domain-predictor-5 found in cache

In [2]:

predict_domain("Mountains are just big rocks.")
# the original return value is under the 'output' key

predict_domain("Mountains are just big rocks.") # the original return value is under the 'output' key

Out[2]:

Trace[str]({ 'created': '2022-07-09T20:38:56.394746',
  'exception': None,
  'feedback': None,
  'logged_values': { 'arg:sentence:length': 29,
                     'arg:sentence:value': 'Mountains are just big rocks.'},
  'models': [{'key': 'my-domain-predictor', 'version': 5}],
  'original_execution_time_ms': 4.999,
  'output': 'geography',
  'tags': ['predict_domain', 'online', 'development'],
  'trace_id': 'aad1f83d-a81f-4b8b-898e-d02f8076616f'})

In [3]:

!great-ai deploy.ipynb
# leave this running and open http://127.0.0.1:6060

!great-ai deploy.ipynb # leave this running and open http://127.0.0.1:6060

2022-07-09 22:38:58 |     INFO | Converting notebook to Python script
2022-07-09 22:38:58 |     INFO | Found `predict_domain` to be the GreatAI app 
2022-07-09 22:38:58 |     INFO | Uvicorn running on http://0.0.0.0:6060 (Press CTRL+C to quit)
2022-07-09 22:39:00 |  WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️
2022-07-09 22:39:00 |  WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver
2022-07-09 22:39:00 |  WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production
2022-07-09 22:39:00 |  WARNING | Cannot find credentials files, defaulting to using LargeFileLocal
2022-07-09 22:39:00 |     INFO | GreatAI (v0.1.0): configured ✅
2022-07-09 22:39:00 |     INFO |   🔩 tracing_database: ParallelTinyDbDriver
2022-07-09 22:39:00 |     INFO |   🔩 large_file_implementation: LargeFileLocal
2022-07-09 22:39:00 |     INFO |   🔩 is_production: False
2022-07-09 22:39:00 |     INFO |   🔩 should_log_exception_stack: True
2022-07-09 22:39:00 |     INFO |   🔩 prediction_cache_size: 512
2022-07-09 22:39:00 |     INFO |   🔩 dashboard_table_size: 20
2022-07-09 22:39:00 |  WARNING | You still need to check whether you follow all best practices before trusting your deployment.
2022-07-09 22:39:00 |  WARNING | > Find out more at https://se-ml.github.io/practices
2022-07-09 22:39:00 |     INFO | Fetching cached versions of my-domain-predictor
2022-07-09 22:39:00 |     INFO | Latest version of my-domain-predictor is 5 (from versions: 0, 1, 2, 3, 4, 5)
2022-07-09 22:39:00 |     INFO | File my-domain-predictor-5 found in cache
2022-07-09 22:39:00 |     INFO | Started server process [882179]
2022-07-09 22:39:00 |     INFO | Waiting for application startup.
2022-07-09 22:39:00 |     INFO | Application startup complete.
^C
2022-07-09 22:39:04 |     INFO | Shutting down
2022-07-09 22:39:04 |     INFO | Waiting for application shutdown.
2022-07-09 22:39:04 |     INFO | Application shutdown complete.
2022-07-09 22:39:04 |     INFO | Finished server process [882179]

Now that you've made sure your application is hardened enough for the intended use case, it is time to deploy it. The responsibilities of GreatAI end when it wraps your inference function and model into a production-ready service. You're given the freedom and responsibility to deploy this service. Fortunately, you (or your organisation) probably already has an established routine for deploying services.

There are three main approaches to deploy a GreatAI service.

Manual deployment¶

Simply run ENVIRONMENT=production great-ai deploy.ipynb in the command-line of a production machine.

This is the crudest approach, however, it might be fitting for some contexts.

Containerised deployment¶

Run the notebook directly in a container or create a service for it using your favourite orchestrator.

docker run \
    -p 6060:6060 \
    --volume `pwd`:/app \
    --rm \
    schmelczera/great-ai deploy.ipynb

You can replace pwd with the path to your code's folder.

Use a Platform-as-a-Service¶

Similarly to the previous approach, your code will run in a container. However, instead of manually managing it, you can just choose from a plethora of PaaS providers that take a Docker image as a source and handle the rest of the deployment.

To this end, you can also create a custom Docker image. It is especially useful if you have third-party dependencies, such as pytorch or tensorflow.

FROM schmelczera/great-ai:latest

# Remove this block if you don't have a requirements.txt
COPY requirements.txt ./   
RUN pip install --no-cache-dir --requirement requirements.txt

# If you store your models in S3 or GridFS, it may be a 
# good idea to # cache them in the image so that you don't
# have to download it each time a container starts
# RUN large-file --backend s3 --secrets s3.ini --cache my-domain-predictor

COPY . .

CMD ["deploy.ipynb"]

Go back to the summary¶

---

Last update: July 9, 2022