Deployed 69f2f6d with MkDocs version: 1.3.1

thesis/chapters/5_cases/naive-bayes.tex

@@ -1,6 +1,6 @@
 \section{Domain classification with Naïve Bayes} \label{section:simple-case}
 
-Using different models for slight variations of the same problem is quite commonplace in the industry. For instance, UberEats has a vast, hierarchical set of models for every country, region, and city for calculating the estimated time of delivery \cite{li2017scaling}. We have also found, that in order to best process an academic publication, knowing its domain is essential. The reason for this can be (among others) the wildly different vocabularies of different domains. For example, the term \textit{framework} in computer science almost always refers to a software artifact (usually implying high tech-transfer potential), while in most other domains, \textit{framework} is used to describe theoretical models that are less central to practical applications. Of course, it is not merely the meaning of the terms but more importantly: their distribution that varies significantly. Therefore, the topic of this section is to design and develop a domain prediction model for academic papers.
+Using different models for slight variations of the same problem is quite commonplace in the industry. For instance, UberEats has a vast, hierarchical set of models for every country, region, and city for calculating the estimated time of delivery \cite{li2017scaling}. We have also found that in order to best process an academic publication, knowing its domain is essential. The reason for this can be (among others) the wildly different vocabularies of different domains. For example, the term \textit{framework} in computer science almost always refers to a software artifact (usually implying high tech-transfer potential), while in most other domains, \textit{framework} is used to describe theoretical models that are less central to practical applications. Of course, it is not merely the meaning of the terms but more importantly: their distribution that varies significantly. Therefore, the topic of this section is to design and develop a domain prediction model for academic papers.
 
 \subsection{Background}
 
@@ -9,7 +9,7 @@ Fortunately, this is one of the oldest text classification tasks. In fact, Maron
 Prior work evaluated SciBERT \cite{beltagy2019scibert} --- a BERT \cite{devlin2018bert} model pretrained on academic publications --- on a simpler version of the task in which the domains of sentences\footnote{Sentences are more appropriate units for processing due to SciBERT's maximum token length of 512 (which comes from its attention mechanism's quadratic complexity \cite{vaswani2017attention}).} have to be decided\footnote{\href{https://paperswithcode.com/sota/sentence-classification-on-paper-field}{paperswithcode.com/sota/sentence-classification-on-paper-field}}. It achieved an F1-score of $0.6571$ after being pretrained on the Semantic Scholar Corpus (SSC) \cite{Lo2020S2ORCTS} and finetuned on the train split of the Microsoft Academic Graph (MAG) dataset \cite{wang2019review}\footnote{SciBERT was applied to a preprocessed version of this dataset available at \href{https://github.com/allenai/scibert/tree/master/data/text_classification/mag}{github.com/allenai/scibert/tree/master/data/text\_classification/mag}}. To the best of my knowledge, no other published work exists on this sentence-classification task. This may be explained by the lack of practical relevance and contrived nature (uniform label distribution) of the task as we will see in the next subsection.
 
 \begin{displayquote}
-\textbf{Design note} After getting familiar with the context, it is time to focus on experimenting and developing our domain prediction service. At the same time, the difficulties encountered should be noted and integrated into GreatAI's design.
+\textbf{Design note} After getting familiar with the context, it is time to focus on experimenting and developing our domain prediction service. At the same time, the difficulties encountered should be noted and integrated into \textit{GreatAI}'s design.
 \end{displayquote}
 
 \subsection{Data}
@@ -43,7 +43,7 @@ SSC is much larger: it contains over 80 million abstracts. Having more data cert
 MAG needs no further preprocessing if we aim to match SciBERT's setup \cite{beltagy2019scibert}. But since SSC contains a heap of metadata, the relevant parts have to be extracted and preprocessed. In this case, these are the concatenation of the abstract's text, paper's title and the journal's name along with the paper's domains (there can be multiple domains for a single paper, it is a multi-label classification task). Lastly, the non-English entries are discarded because we only expect to process papers in  English.
 
 \begin{displayquote}
-\textbf{How should we preprocess the data?} These simple processing steps (filter, map, project) are almost always present in the data science life-cycle. For example, cleaning the input text from various HTML, OCR, PDF, or \LaTeX \hskip 0.12cm extraction artifacts is almost always necessary for text analysis. This is captured in the AI best practices collection under the following category: \textit{Write Reusable Scripts for Data Cleaning and Merging}. Also, the best practice of \textit{Test all Feature Extraction Code} is somewhat applicable: the applied processing steps must not introduce unwanted artifacts.
+\textbf{How should we preprocess the data?} These simple processing steps (filter, map, project) are almost always present in the data science lifecycle. For example, cleaning the input text from various HTML, OCR, PDF, or \LaTeX \hskip 0.12cm extraction artifacts is almost always necessary for text analysis. This is captured in the AI best practices collection under the following category: \textit{Write Reusable Scripts for Data Cleaning and Merging}. Also, the best practice of \textit{Test all Feature Extraction Code} is somewhat applicable: the applied processing steps must not introduce unwanted artifacts.
 \end{displayquote}
 
 \subsection{Methods}
@@ -57,7 +57,7 @@ Using the MNB implementation of scikit-learn \cite{pedregosa2011scikit}, it only
 The sentences are tokenised into words and vectorised with TF-IDF (with logarithmic term frequency) \cite{buckley1985implementation}, the hyperparameters found via 3-fold cross-validation on the \textit{train} split lead to filtering out tokens which occur in fewer than 5 documents or more than 5\% of the documents.
 
 \begin{displayquote}
-\textbf{What could be automated here?} As discussed in Section \ref{section:accessible-ai}, libraries exposing algorithms and state-of-the-art models can already be considered mature and accessible. In this case, only scikit-learn was utilised, but subjectively, most popular libraries have a similarly easy to use use API. Therefore, I see no urgent need for further action regarding the \textit{experimentation} step of the life-cycle in connection with the AI best practices.
+\textbf{What could be automated here?} As discussed in Section \ref{section:accessible-ai}, libraries exposing algorithms and state-of-the-art models can already be considered mature and accessible. In this case, only scikit-learn was utilised, but subjectively, most popular libraries have a similarly easy to use use API. Therefore, I see no urgent need for further action regarding the \textit{experimentation} step of the lifecycle in connection with the AI best practices.
 \end{displayquote}
 
 \subsection{Results \& Discussion}
@@ -95,7 +95,7 @@ This is the usual point where papers conclude: a proof-of-concept/prototype has
 First, an inference function needs to be written that can take an input on the fly and calculate a corresponding prediction. Since we aim to follow the best practices, namely: \textit{Explain Results and Decisions to Users} and \textit{Employ Interpretable Models When Possible}, giving an explanation of the results is expected. Fortunately, with our simple model it is easy to determine the most influential weights, thus, words; the explanations are derived by taking the top 5 tokens from the input text ranked by their feature weights. The last deployment step may be to provide access to our model for others.
 
 \begin{displayquote}
-\textbf{How do we provide an interface for the inference function?} We either have an offline or online inference workflow (or both). For the former, we have to provide a way to use it in batch processing; a simple Python function may be adequate for this purpose, though, allowing it to be easily (or automatically) parallelised would make its consumers' DX better. If it is an online-workflow, we must have a service running continuously and accepting input at any time. This can be achieved by a remote procedure call (RPC) interface, or more commonly, a web API. Developers usually refer to these as REST API-s, sometimes, they even follow the conventions of REST. Either way, we must develop a wrapper over the service in order to make it available for other internal/external consumers.
+\textbf{How do we provide an interface for the inference function?} We either have an offline or online inference workflow (or both). For the former, we have to provide a way to use it in batch processing; a simple Python function may be adequate for this purpose, though, allowing it to be easily (or automatically) parallelised would make its consumers' DX better. If it is an online-workflow, we must have a service running continuously and accepting input at any time. This can be achieved by a remote procedure call (RPC) interface, or more commonly, a web API. Developers usually refer to these as REST APIs, sometimes, they even follow the conventions of REST. Either way, we must develop a wrapper over the service in order to make it available for other internal/external consumers.
 \end{displayquote}
 
 According to the body of research on the adoption of best practices, this is where many real-world projects conclude. This also happens to be \textbf{the gap}. Believing that solely focusing on the research and experiments is good enough is a fallacy: when following this approach, the deployment step ends up being a rushed attempt of wrapping the \textit{AI} and putting it in the production environment. This is inarguably a deployment. However, it likely follows very few of the best practices which can lead to suboptimal real-life performance, lack of accountability, lack of opportunity to improve, and possibly an overall negative societal impact.
@@ -106,9 +106,9 @@ According to the body of research on the adoption of best practices, this is whe
 
 \section{Bridging the gap with GreatAI}
 
-First, let us revisit the library's scope for clarification. As concluded in Section \ref{section:scope}, GreatAI should ease the \textit{transition} step between prototypes and production-ready deployments. However, this leaves open the question of what constitutes to this step? There are cross-cutting concerns, for example, feature extraction is implemented and used in the training phase but it is also deployed alongside the model. The robustness criterion has to be met by this procedure even though its implementation is only in focus in the earlier stages of the project. Since having an untested function deployed into production can have severe repercussions, I conclude, assuring its correctness lies within the scope of GreatAI.
+First, let us revisit the library's scope for clarification. As concluded in Section \ref{section:scope}, \textit{GreatAI} should ease the \textit{transition} step between prototypes and production-ready deployments. However, this leaves open the question of what constitutes to this step? There are cross-cutting concerns, for example, feature extraction is implemented and used in the training phase but it is also deployed alongside the model. The robustness criterion has to be met by this procedure even though its implementation is only in focus in the earlier stages of the project. Since having an untested function deployed into production can have severe repercussions, I conclude, assuring its correctness lies within the scope of \textit{GreatAI}.
 
-This section briefly explores how the problems raised can be solved using GreatAI, and the API it provides in order to best fit the needs of its users. We first focus on the aspects of data, then, the automated wrapping of services, lastly we discuss the utility of helper functions.
+This section briefly explores how the problems raised can be solved using \textit{GreatAI}, and the API it provides in order to best fit the needs of its users. We first focus on the aspects of data, then, the automated wrapping of services, lastly we discuss the utility of helper functions.
 
 \subsection{Handling data} \label{subsection:large-file}
 
@@ -118,7 +118,7 @@ There are two kinds of data storage needs we have to address: training data and
 
 An open-source tool, the Data Version Control (DVC)\footnote{\href{https://dvc.org/}{dvc.org}} provides a nearly perfect alternative. It comes with a command-line interface (CLI) inspired by Git's, and it can be integrated with several backend storage servers. Its only downside is, of course, that it is one more tool that increases the complexity of the project and the initial setup time. If this is an acceptable price to pay, then I personally recommend opting for DVC. Nevertheless, if this may prohibit a team\footnote{As was the case with MLFlow tracking in an ING team that we saw in Section \ref{section:industry}.} from properly handling data according to the best practices, I present a simpler solution.
 
-The complexity of an API can be decreased by relying on its users preexisting knowledge and known patterns \cite{hermans2021programmer,ousterhout2018philosophy}. Therefore, we can reuse familiar API-s, such as the \texttt{open()} method from Python. Therefore, a method is proposed which provides the same interface, however, the backing storage can be a mixture of local disk space, S3-compatible storage, MongoDB, or any other storage backend. It provides a superset of \texttt{open()}'s interface\footnote{\href{https://docs.python.org/3/library/functions.html\#open}{docs.python.org/3/library/functions.html\#open}}; the same parameters can be used with it resulting in similar observed behaviour.
+The complexity of an API can be decreased by relying on its users preexisting knowledge and known patterns \cite{hermans2021programmer,ousterhout2018philosophy}. Therefore, we can reuse familiar APIs, such as the \texttt{open()} method from Python. Therefore, a method is proposed which provides the same interface, however, the backing storage can be a mixture of local disk space, S3-compatible storage, MongoDB, or any other storage backend. It provides a superset of \texttt{open()}'s interface\footnote{\href{https://docs.python.org/3/library/functions.html\#open}{docs.python.org/3/library/functions.html\#open}}; the same parameters can be used with it resulting in similar observed behaviour.
 
 The expected features: versioning, progress bars, caching, garbage collecting the cache, automatically deleting old remote version are all present and come with recommended --- but easy to see and change --- configuration.
 
@@ -130,7 +130,7 @@ Some of the expectations one might have for data-intensive (such as AI) software
 
 Inline with the findings of John et al. \cite{john2020architecting} on the SOTA of AI deployments, I suggest we wrap the applications in a format which is more compatible with existing DevOps tool-kits. Instead of reinventing the wheel, we should rely on more established DevOps best practices for implementing the SE4ML deployment best practices. Besides, organisations are expected to have their deployment processes for classical applications, thus, allowing them to reuse those for AI applications seems to be the most convenient approach.
 
-Based on personal empirical evidence, three types of software artifacts are identified (in the context of Python) for which a wide range of established practices exist. WSGI server\footnote{\href{https://peps.python.org/pep-3333/}{peps.python.org/pep-3333}} compatible applications, executable scripts, and Docker Images\footnote{\href{https://docs.docker.com/registry/spec/manifest-v2-2/}{docs.docker.com/registry/spec/manifest-v2-2}}. To achieve this, GreatAI provides a compatibility layer between simple Python inference functions and all the above common artifacts. Taking functions as input for the first step is inline with the requirement to be \textbf{General}. Nevertheless, in order to also allow customisation, additional configuration, metadata, and behavioural specification can be given as well.
+Based on personal empirical evidence, three types of software artifacts are identified (in the context of Python) for which a wide range of established practices exist. WSGI server\footnote{\href{https://peps.python.org/pep-3333/}{peps.python.org/pep-3333}} compatible applications, executable scripts, and Docker Images\footnote{\href{https://docs.docker.com/registry/spec/manifest-v2-2/}{docs.docker.com/registry/spec/manifest-v2-2}}. To achieve this, \textit{GreatAI} provides a compatibility layer between simple Python inference functions and all the above common artifacts. Taking functions as input for the first step is inline with the requirement to be \textbf{General}. Nevertheless, in order to also allow customisation, additional configuration, metadata, and behavioural specification can be given as well.
 
 The main advantage of the wrapping approach is that it does not require any input from the clients (by default). I opted for a decorator which lets users wrap their function by adding a single additional line of code as shown in Listing \ref{listing:hello-world}.  After which the created WSGI application can be accessed through the \texttt{greeter.app} property. A CLI script (\texttt{great-ai}), along with a \texttt{Dockerfile} are also provided to cover the other two deployment artifacts.
 
@@ -147,7 +147,7 @@ from great_ai import GreatAI
 def greeter(name: str) -> str:
     return f"Hello {name}!"
 \end{minted}
-\caption{Simplest example using GreatAI for wrapping a function. In practice, \texttt{greeter} probably would be the inference function of an ML model.}
+\caption{Simplest example using \textit{GreatAI} for wrapping a function. In practice, \texttt{greeter} probably would be the inference function of an ML model.}
 \label{listing:hello-world}
 \end{listing}
 
@@ -176,7 +176,7 @@ def add_to_secret_number(positive_number: int, model: int) -> int:
 
 assert add_number(1).output == 5
 \end{minted}
-\caption{A simple GreatAI service with behavioural customisations. In practice, the function would probably be the inference function for an ML model.}
+\caption{A simple \textit{GreatAI} service with behavioural customisations. In practice, the function would probably be the inference function for an ML model.}
 \label{listing:complex}
 \end{listing}
 
@@ -192,7 +192,7 @@ A function called \texttt{parallel\_map()} is implemented which closely mimics t
 
 \subsection{Summary}
 
-After implementing some features of the library it can be already used for deploying the previously discussed domain prediction model. In this case, online prediction is expected, hence, the REST API-based deployment is chosen which is created by GreatAI and packaged in a Docker image. This image can be instantiated by the company's existing DevOps pipeline and cloud infrastructure. At the end, users can see one more tag in the header section of publication evaluations where they can also see the explanation behind the model's decision as seen in Figure \ref{fig:dashboard-domains}. Let us now explore how it fares in a more complex case.
+After implementing some features of the library it can be already used for deploying the previously discussed domain prediction model. In this case, online prediction is expected, hence, the REST API-based deployment is chosen which is created by \textit{GreatAI} and packaged in a Docker image. This image can be instantiated by the company's existing DevOps pipeline and cloud infrastructure. At the end, users can see one more tag in the header section of publication evaluations where they can also see the explanation behind the model's decision as seen in Figure \ref{fig:dashboard-domains}. Let us now explore how it fares in a more complex case.
 
 \begin{figure}
     \centering

thesis/chapters/5_cases/scibert.tex

@@ -5,7 +5,7 @@ Let us now turn our attention towards a more complex component. The ScoutinScien
 The current approach uses a simple heuristic based on a set of phrases selected by business developers and extended by the help of a word2vec model \cite{mikolov2013efficient}. The user feedback deemed this implementation slightly helpful but not adequate for providing an accurate overview. Thus, this is the baseline that I attempt to improve on in this section.
 
 \begin{displayquote}
-Compared with Section \ref{section:simple-case}, this time around, the toolset of GreatAI is available at our disposal. Hopefully, this will streamline the development and --- especially --- the deployment. Given its arguably higher complexity, the experiment falls closer to industrial use-cases, and hence, can give more accurate feedback on how to further improve the API.
+Compared with Section \ref{section:simple-case}, this time around, the toolset of \textit{GreatAI} is available at our disposal. Hopefully, this will streamline the development and --- especially --- the deployment. Given its arguably higher complexity, the experiment falls closer to industrial use-cases, and hence, can give more accurate feedback on how to further improve the API.
 \end{displayquote}
 
 \subsection{Background}
@@ -43,11 +43,11 @@ For the first iteration, 1500 sentences were selected for 2 experts to annotate
 The next step is finetuning SciBERT with the help of HuggingFace transformers \cite{wolf2019huggingface}. The data are divided into training and test splits with a ratio of 4:1. From the train split, a validation split is also derived which is used for early stopping. The objective function is the positive class' F1-score and the early stopping patience is 5 epochs. The learning rate is $5 \times 10^{-5}$ and AdamW \cite{loshchilov2017decoupled} is used for optimisation with a weight decay of 0.05. The code can be found in the documentation\footnote{\href{https://great-ai.scoutinscience.com/examples/scibert/train/}{great-ai.scoutinscience.com/examples/scibert/train/}}, it is surprisingly slightly shorter than the code of Section \ref{section:simple-case}.
 
 \begin{displayquote}
-\textbf{Reproducability} Reproducible experiments are generally preferred. It is easy to forget to set some seeds values and, for example, end up with different datapoints in the test-train splits during training and validation in a Continuous Integration (CI) pipeline. To facilitate reproducability, it would be useful to reset the seeds of each imported library's random number generators (RNG) when GreatAI is configured. Thus, a feature has been added to detect and reset RNG-s of installed and imported libraries. This certainly will not solve the reproducibility crisis \cite{hutson2018artificial} on its own, however, in some cases, it can result in one fewer step to miss.
+\textbf{Reproducability} Reproducible experiments are generally preferred. It is easy to forget to set some seeds values and, for example, end up with different datapoints in the test-train splits during training and validation in a Continuous Integration (CI) pipeline. To facilitate reproducability, it would be useful to reset the seeds of each imported library's random number generators (RNGs) when \textit{GreatAI} is configured. Thus, a feature has been added to detect and reset RNGs of installed and imported libraries. This certainly will not solve the reproducibility crisis \cite{hutson2018artificial} on its own, however, in some cases, it can result in one fewer step to miss.
 \end{displayquote}
 
 \begin{displayquote}
-\textbf{Utility of LargeFiles-s} For the purposes of the documentation, the finetuning was conducted in the Google Colab online environment which is excellent for providing anyone with GPU-time for free. However, notebook environments are ephemeral, resulting in the need to manually upload and download all relevant data whenever a new virtual machine (VM) instance is granted. The \texttt{LargeFile} implementation alleviated this problem by handling the uploads and downloads automatically. Of course, first, backwards compatibility had to be solved for Python 3.7 which is the only available environment in Colab.
+\textbf{Utility of LargeFiles} For the purposes of the documentation, the finetuning was conducted in the Google Colab online environment which is excellent for providing anyone with GPU-time for free. However, notebook environments are ephemeral, resulting in the need to manually upload and download all relevant data whenever a new virtual machine (VM) instance is granted. The \texttt{LargeFile} implementation alleviated this problem by handling the uploads and downloads automatically. Of course, first, backwards compatibility had to be solved for Python 3.7 which is the only available environment in Colab.
 \end{displayquote}
 
 The best validation results were achieved after 8 epochs which was slightly more than expected but is presumably due to the weight decay. The confusion matrix on the test split can be seen in Figure \ref{fig:scibert-confusion}: regardless the subjective definition of the task, SciBERT manages to achieve good quality which is indicated by an F1-score of \textbf{0.89}.
@@ -96,7 +96,7 @@ Even though the operation is conceptually simple, because of the opaque datastru
 
 \section{Improving GreatAI}
 
-After having solved two problems by implementing two standalone services and integrating them into an existing ecosystem while relying on GreatAI as a primary tool, a wide variety of insights have been gained. In the next couple of subsections, the extra features and design decisions are presented that have been motivated by the \textit{Highlights service}. After which, the final surface of the API is described and evaluated by its relation to the SE4ML \cite{serban2020adoption,serban2021practices} and AI engineering \cite{john2020architecting,john2020ai} best practices.
+After having solved two problems by implementing two standalone services and integrating them into an existing ecosystem while relying on \textit{GreatAI} as a primary tool, a wide variety of insights have been gained. In the next couple of subsections, the extra features and design decisions are presented that have been motivated by the \textit{Highlights service}. After which, the final surface of the API is described and evaluated by its relation to the SE4ML \cite{serban2020adoption,serban2021practices} and AI engineering \cite{john2020architecting,john2020ai} best practices.
 
 \subsection{Caching}
 
@@ -104,13 +104,13 @@ Sustainability is an increasingly crucial concern of ethical AI \cite{van2021sus
 
 \subsection{Revisiting \texttt{parallel\_map}}
 
-Even though most inference functions are CPU-bound, turns out, sometimes they involve IO, especially, when relying on the results of other, remote models. This means that a significant performance improvement can be achieved by implementing some inference functions asynchronously \cite{tilkov2010node}. Thus, GreatAI also has to support decorating both regular (synchronous) and asynchronous functions. There is one notable consequence of this: the batch processing feature also has to be compatible with \texttt{async} inference functions. Batch processing is still a useful feature since it is likely that async inference functions are both IO (remote calls) and CPU (local evaluation) constrained at the same time, thus, they can benefit from multi-core parallelisation. 
+Even though most inference functions are CPU-bound, turns out, sometimes they involve IO, especially, when relying on the results of other, remote models. This means that a significant performance improvement can be achieved by implementing some inference functions asynchronously \cite{tilkov2010node}. Thus, \textit{GreatAI} also has to support decorating both regular (synchronous) and asynchronous functions. There is one notable consequence of this: the batch processing feature also has to be compatible with \texttt{async} inference functions. Batch processing is still a useful feature since it is likely that async inference functions are both IO (remote calls) and CPU (local evaluation) constrained at the same time, thus, they can benefit from multi-core parallelisation. 
 
 However, the standard library's \texttt{multiprocessing}, the third party \texttt{multiprocess} \cite{mckerns2012building}, and, another popular library, \texttt{joblib}\footnote{\href{https://joblib.readthedocs.io/en/latest/}{joblib.readthedocs.io/en/latest}} all lack the support for efficiently parallelising async functions. For this reason, \texttt{parallel\_map} is reimplemented to create an event-loop in each worker process to keep the efficiency of non-blocking IO while also providing parallelisation for the CPU-bound sections of code.
 
 \subsection{Programmatic integration}
 
-Apart from supporting \texttt{async} calls, there are a couple of more step that can be taken to help integrating any service with a GreatAI deployment. This is implemented by the \texttt{call\_remote\_great\_ai} function which hides the networking required to call a GreatAI instance's REST API. It takes care of validation, automatic retries, serialisation, and deserialisation. This comes with the added benefit of encouraging decoupled services because the friction of integrating them is no longer noticeable which is beneficial for human collaboration \cite{hasselbring2002component}.
+Apart from supporting \texttt{async} calls, there are a couple of more step that can be taken to help integrating any service with a \textit{GreatAI} deployment. This is implemented by the \texttt{call\_remote\_great\_ai} function which hides the networking required to call a \textit{GreatAI} instance's REST API. It takes care of validation, automatic retries, serialisation, and deserialisation. This comes with the added benefit of encouraging decoupled services because the friction of integrating them is no longer noticeable which is beneficial for human collaboration \cite{hasselbring2002component}.
 
 Additionally, a REST API is generated with its accompanying OpenAPI schema\footnote{\href{https://swagger.io/specification}{swagger.io/specification}} and served with a \href{https://swagger.io/}{Swagger} template. It also contains metadata about the function, for instance, its docstring, version, and version of its registered models concatenated in order to be SemVer\footnote{\href{https://semver.org/}{semver.org}} compatible. These can be seen in Figure \ref{fig:greatai-api}. This, combined with a \texttt{/version} HTTP endpoint for programmatic access and validation of the service's metadata proved to be key features when integrating the \textit{Highlights service} into ScoutinScience's service-based architecture.
 
@@ -118,13 +118,13 @@ Additionally, a REST API is generated with its accompanying OpenAPI schema\footn
     \centering
     \includegraphics[width=0.85\linewidth]{figures/greatai-api.png}
     \captionsetup{width=.9\linewidth}
-    \caption{Documentation of the automatically scaffolded REST API of a GreatAI service. Notice, how its version string includes its registered models in a SemVer compliant way: \texttt{0.0.1+small-domain-prediction-v11}.}
+    \caption{Documentation of the automatically scaffolded REST API of a \textit{GreatAI} service. Notice, how its version string includes its registered models in a SemVer compliant way: \texttt{0.0.1+small-domain-prediction-v11}.}
     \label{fig:greatai-api}
 \end{figure}
 
 \subsection{\textit{Human} integration}
 
-Even though the REST API of GreatAI services exposes all necessary features\footnote{Such as providing feedback per prediction, complexly filtering and sorting traces, create-read-update-delete (CRUD) operations for the feedback and traces, accessing live monitoring info (current configuration, versions, cache statistics), etc.} which are great for programmatic access, these are not ideal for direct human comprehension. In order to ease the introduction of GreatAI services, a rudimentary dashboard is --- optionally --- generated next to the REST API. The dashboard's main features can be observed in Figures \ref{fig:greatai-header}, \ref{fig:greatai-table}, and \ref{fig:greatai-parallel}. The diagrams and filterable/sortable table are interconnected and are automatically updated, the reactive behaviour is provided by the Dash framework \cite{shammamah_hossain-proc-scipy-2019}.
+Even though the REST API of \textit{GreatAI} services exposes all necessary features\footnote{Such as providing feedback per prediction, complexly filtering and sorting traces, create-read-update-delete (CRUD) operations for the feedback and traces, accessing live monitoring info (current configuration, versions, cache statistics), etc.} which are great for programmatic access, these are not ideal for direct human comprehension. In order to ease the introduction of \textit{GreatAI} services, a rudimentary dashboard is --- optionally --- generated next to the REST API. The dashboard's main features can be observed in Figures \ref{fig:greatai-header}, \ref{fig:greatai-table}, and \ref{fig:greatai-parallel}. The diagrams and filterable/sortable table are interconnected and are automatically updated, the reactive behaviour is provided by the Dash framework \cite{shammamah_hossain-proc-scipy-2019}.
 
 \begin{figure}
     \centering