Update thesis with first feedback from @jstvssr
This commit is contained in:
parent
37b51428fb
commit
b2c76c78c1
20 changed files with 1013 additions and 23 deletions
|
|
@ -1,6 +1,6 @@
|
|||
\chapter{The ScoutinScience platform} \label{chapter:case}
|
||||
|
||||
The core product of \href{https://scoutinscience.com/}{ScoutinScience B.V.} is its platform\footnote{\href{https://dashboard.scoutinscience.com/}{dashboard.scoutinscience.com}}. The clients are technology-transfer offices of Dutch and German universities, government organisations (e.g. Wetsus), and corporates (e.g. Heraeus Group and Ruma Rubber B.V.) who wish to extend the scope of their R\&D activities. ScoutinScience connects to multiple data sources of academic publications and integrates them into a single database. Each new publication is evaluated with a suite of AI components that ultimately determine its technology transfer potential. Other markers are also extracted that help the users get a quick overview of the authors, topics, and contributions of a given piece of research.
|
||||
The core product of ScoutinScience B.V.\footnote{\href{https://scoutinscience.com/}{scoutinscience.com}} is its platform\footnote{\href{https://dashboard.scoutinscience.com/}{dashboard.scoutinscience.com}}. The clients are technology-transfer offices of Dutch and German universities, government organisations (e.g. Wetsus), and corporates (e.g. Heraeus Group and Ruma Rubber B.V.) who wish to extend the scope of their R\&D activities. ScoutinScience connects to multiple data sources of academic publications and integrates them into a single database. Each new publication is evaluated with a suite of AI components that ultimately determine its technology transfer potential. Other markers are also extracted that help the users get a quick overview of the authors, topics, and contributions of a given piece of research.
|
||||
|
||||
Each client organisation gets to see a different filtered view of this database ranked by the predicted probability of technology transfer opportunities being present. The main motivation is to make these business developers' and other professionals' work more efficient by showing them which papers have the highest chance of being considered interesting by them.
|
||||
|
||||
|
|
|
|||
|
|
@ -82,7 +82,7 @@ When this model is applied to the \textit{test} split of MAG, we get the confusi
|
|||
|
||||
It is, of course, not entirely surprising that the sophisticated transformer architecture of SciBERT is not necessary for a straightforward task like this. Apart from phrases, the relations between separate words of a sentence do not carry nearly as much discriminative power as the identity of the terms \cite{hand2001idiot}; hence, there is little reason for using an attention mechanism. The fact that SciBERT even works in any way on this task is already a testament to its general applicability. Nevertheless, this short experiment has proved that we can safely opt for using MNB for production.
|
||||
|
||||
Since Multinomial Naïve Bayes is best at returning a single label and SSC has multiple labels per datapoint: for evaluation purposes, it is checked whether the returned label is contained in the labels of the ground truth. On this dataset, MNB achieves a lower macro-average F1-score of 0.59.\footnote{The code for this is available at \href{https://great-ai.scoutinscience.com/examples/simple/deploy}{great-ai.scoutinscience.com/examples/simple/deploy}.} The weighted-average F1 is 0.70, and the overall accuracy is also 70\%. The substantial difference between the macro and weighted averages comes from the unbalanced distribution of the labels.
|
||||
Since Multinomial Naïve Bayes is best at returning a single label and SSC has multiple labels per datapoint: for evaluation purposes, it is checked whether the returned label is contained in the labels of the ground truth. On this dataset, MNB achieves a lower macro-average than on MAG, with F1-score of 0.59.\footnote{The code for this is available at \href{https://great-ai.scoutinscience.com/examples/simple/deploy}{great-ai.scoutinscience.com/examples/simple/deploy}.} The weighted-average F1 is 0.70, and the overall accuracy is also 70\%. The substantial difference between the macro and weighted averages comes from the unbalanced distribution of the labels.
|
||||
|
||||
The lower F1-score is not surprising because this dataset has more than twice as many classes. Additionally, the mistakes made are defensible when we look at Figure \ref{fig:ss-confusion}: most of them are between close or related classes.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
\newpage
|
||||
\section{Text summarisation with SciBERT} \label{section:complex-case}
|
||||
|
||||
The ScoutinScience Dashboard contains a full-page evaluation view for academic publications. On this, the known metadata, historical trends about the paper's topics, social media mentions, a PDF viewer showing the document, and other augmentation tools are displayed. One of these is the \textit{Highlights} section, which aims to summarise the paper from a technology-transfer perspective.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue