andras/great-ai
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 4 Packages Wiki Activity Actions

Deployed b5a69fe with MkDocs version: 1.3.1

Browse source
This commit is contained in:
2022-08-20 10:59:19 +00:00
parent 3cf6b08552
commit fbc107f453
25 changed files with 169 additions and 170 deletions
Download patch file Download diff file Expand all files Collapse all files

10
how-to-guides/handle-training-data/handle-training-data.md
View file

@ -1,13 +1,13 @@
# How to manage training data
In order to simplify your training data management, `great-ai` provide two complementing approaches for inputting new data-points.
In order to simplify your training data management, `great-ai` provide two complementing approaches for inputting new data points.
## Upload data
At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call [great_ai.add_ground_truth][]. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, [great_ai.query_ground_truth][] can be called to get a filtered view of the training data.
!!! important "Train-test-validation splits"
It is a best practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.
It is a best practice to lock away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results; hence, care must be taken to avoid it.
With [great_ai.add_ground_truth][], there is an option to tag the samples with `train`, `test`, and `validation` randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.
@ -25,7 +25,7 @@ add_ground_truth(
)
```
1. Note that the ratios don't have to add up to 1. They are just weights. There is also a `validation_split_ratio` which is 0 by default.
1. Note that the ratios don't have to add up to 1. They are just weights. There is also a `validation_split_ratio` which is 0 by default.
```python
>>> from great_ai import query_ground_truth
@ -50,13 +50,13 @@ add_ground_truth(
'trace_id': 'abee0671-beb9-4284-8c3b-c65e5836ce38'})]
```
1. Expected output. This can be also accessed through the `.output` property.
1. Expected output. This can also be accessed through the `.output` property.
2. The input value is stored here.
3. Notice how `ground_truth` is always included as a tag when using [great_ai.add_ground_truth][].
## Get feedback
After the initial data gathering, end-to-end feedback can be also integrated into the dataset.
After the initial data gathering, end-to-end feedback can also be integrated into the dataset.
The scaffolded REST API contains endpoints for managing traces and their feedbacks.