andras/great-ai
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 4 Packages Wiki Activity Actions

Deployed aa31d2b with MkDocs version: 1.3.0

Browse source
This commit is contained in:
2022-07-15 15:52:07 +00:00
parent 5a8662103a
commit 087e049514
17 changed files with 58 additions and 64 deletions
Download patch file Download diff file Expand all files Collapse all files

6
tutorial/train/train.ipynb
View file

@ -30,9 +30,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"First, we have to get some data. After downloading it [from here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag), we might notice that the dataset is in [JSON Lines](https://jsonlines.org/) format (each line is a seperate JSON document). \n",
"First, we have to get some data. After downloading it [from here](https://github.com/allenai/scibert/tree/master/data/text_classification/mag), we might notice that the dataset is in [JSON Lines](https://jsonlines.org/) format (each line is a separate JSON document). \n",
"\n",
"Let's write a function which takes a single line, and returns the sentence and the corresponding label from it. Before returning, the sentence is also [cleaned](/reference/utilities/#great_ai.utilities.clean.clean) to remove any LaTeX, XML, unicode, PDF-extraction artifacts."
"Let's write a function which takes a single line and returns the sentence and the corresponding label from it. Before returning, the sentence is also [cleaned](/reference/utilities/#great_ai.utilities.clean.clean) to remove any LaTeX, XML, unicode, PDF-extraction artifacts."
]
},
{
@ -238,7 +238,7 @@
"\n",
"You might wonder that *\"this is great, but besides some utility functions (`clean`, `simple_parallel_map`, ...) what more value does GreatAI add?\"*. This would be a valid argument because the scope of GreatAI actually only starts here.\n",
"\n",
"> Not coincidentally, this is the point where the scope of Data Science ends but it's still a grey-zone for software engineering.\n",
"> Not coincidentally, this is the point where the scope of Data Science ends but it's still a grey zone for software engineering.\n",
"\n",
"In order to use this model in production, we have to make it available on some possibly shared infrastructure."
]