1811 lines
61 KiB
Text
1811 lines
61 KiB
Text
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus)\n",
|
|
"\n",
|
|
"> Part 2: train a model\n",
|
|
"\n",
|
|
"\n",
|
|
"> The blue boxes show the steps implemented in this notebook.\n",
|
|
"\n",
|
|
"In [Part 1](data.ipynb), we have cleaned and transformed our training data. We can now access this data using `great_ai.LargeFile`. Locally, it will gives us the cached version, otherwise, the latest version is downloaded from S3. \n",
|
|
"\n",
|
|
"In this part, we hyperparameter-optimise and train a simple, Naive Bayes classifier which we then export for deployment using `great_ai.save_model`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Load data that has been extracted in [part 1](data.ipynb)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\u001b[38;5;226m2022-06-25 14:50:29,879 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,880 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising MongodbDriver\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,881 | INFO | Found credentials file (/data/projects/great_ai_example/mongo.ini), initialising LargeFileMongo\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,881 | INFO | Settings: configured ✅\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,882 | INFO | 🔩 tracing_database: MongodbDriver\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,883 | INFO | 🔩 large_file_implementation: LargeFileMongo\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,883 | INFO | 🔩 is_production: False\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,884 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,884 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:29,885 | INFO | 🔩 dashboard_table_size: 50\u001b[0m\n",
|
|
"\u001b[38;5;226m2022-06-25 14:50:29,885 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
|
|
"\u001b[38;5;226m2022-06-25 14:50:29,885 | WARNING | > Find out more at https://se-ml.github.io/practices/\u001b[0m\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from great_ai import query_ground_truth\n",
|
|
"\n",
|
|
"data = query_ground_truth(\"train\")\n",
|
|
"X = [d.input for d in data for domain in d.feedback]\n",
|
|
"y = [domain for d in data for domain in d.feedback]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.plotly.v1+json": {
|
|
"config": {
|
|
"plotlyServerURL": "https://plot.ly"
|
|
},
|
|
"data": [
|
|
{
|
|
"alignmentgroup": "True",
|
|
"hovertemplate": "domain=%{x}<br>count=%{y}<extra></extra>",
|
|
"legendgroup": "",
|
|
"marker": {
|
|
"color": "#636efa",
|
|
"pattern": {
|
|
"shape": ""
|
|
}
|
|
},
|
|
"name": "",
|
|
"offsetgroup": "",
|
|
"orientation": "v",
|
|
"showlegend": false,
|
|
"textposition": "auto",
|
|
"type": "bar",
|
|
"x": [
|
|
"Medicine",
|
|
"Computer Science",
|
|
"Chemistry",
|
|
"Biology",
|
|
"Materials Science",
|
|
"Engineering",
|
|
"Physics",
|
|
"Psychology",
|
|
"Mathematics",
|
|
"Sociology",
|
|
"Business",
|
|
"Political Science",
|
|
"Economics",
|
|
"Geology",
|
|
"Geography",
|
|
"Environmental Science",
|
|
"History",
|
|
"Art",
|
|
"Philosophy"
|
|
],
|
|
"xaxis": "x",
|
|
"y": [
|
|
4200,
|
|
1371,
|
|
1307,
|
|
1300,
|
|
1111,
|
|
876,
|
|
639,
|
|
617,
|
|
569,
|
|
334,
|
|
323,
|
|
311,
|
|
269,
|
|
256,
|
|
245,
|
|
244,
|
|
143,
|
|
137,
|
|
62
|
|
],
|
|
"yaxis": "y"
|
|
}
|
|
],
|
|
"layout": {
|
|
"barmode": "relative",
|
|
"height": 400,
|
|
"legend": {
|
|
"tracegroupgap": 0
|
|
},
|
|
"margin": {
|
|
"t": 60
|
|
},
|
|
"template": {
|
|
"data": {
|
|
"bar": [
|
|
{
|
|
"error_x": {
|
|
"color": "#2a3f5f"
|
|
},
|
|
"error_y": {
|
|
"color": "#2a3f5f"
|
|
},
|
|
"marker": {
|
|
"line": {
|
|
"color": "#E5ECF6",
|
|
"width": 0.5
|
|
},
|
|
"pattern": {
|
|
"fillmode": "overlay",
|
|
"size": 10,
|
|
"solidity": 0.2
|
|
}
|
|
},
|
|
"type": "bar"
|
|
}
|
|
],
|
|
"barpolar": [
|
|
{
|
|
"marker": {
|
|
"line": {
|
|
"color": "#E5ECF6",
|
|
"width": 0.5
|
|
},
|
|
"pattern": {
|
|
"fillmode": "overlay",
|
|
"size": 10,
|
|
"solidity": 0.2
|
|
}
|
|
},
|
|
"type": "barpolar"
|
|
}
|
|
],
|
|
"carpet": [
|
|
{
|
|
"aaxis": {
|
|
"endlinecolor": "#2a3f5f",
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"minorgridcolor": "white",
|
|
"startlinecolor": "#2a3f5f"
|
|
},
|
|
"baxis": {
|
|
"endlinecolor": "#2a3f5f",
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"minorgridcolor": "white",
|
|
"startlinecolor": "#2a3f5f"
|
|
},
|
|
"type": "carpet"
|
|
}
|
|
],
|
|
"choropleth": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"type": "choropleth"
|
|
}
|
|
],
|
|
"contour": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"colorscale": [
|
|
[
|
|
0,
|
|
"#0d0887"
|
|
],
|
|
[
|
|
0.1111111111111111,
|
|
"#46039f"
|
|
],
|
|
[
|
|
0.2222222222222222,
|
|
"#7201a8"
|
|
],
|
|
[
|
|
0.3333333333333333,
|
|
"#9c179e"
|
|
],
|
|
[
|
|
0.4444444444444444,
|
|
"#bd3786"
|
|
],
|
|
[
|
|
0.5555555555555556,
|
|
"#d8576b"
|
|
],
|
|
[
|
|
0.6666666666666666,
|
|
"#ed7953"
|
|
],
|
|
[
|
|
0.7777777777777778,
|
|
"#fb9f3a"
|
|
],
|
|
[
|
|
0.8888888888888888,
|
|
"#fdca26"
|
|
],
|
|
[
|
|
1,
|
|
"#f0f921"
|
|
]
|
|
],
|
|
"type": "contour"
|
|
}
|
|
],
|
|
"contourcarpet": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"type": "contourcarpet"
|
|
}
|
|
],
|
|
"heatmap": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"colorscale": [
|
|
[
|
|
0,
|
|
"#0d0887"
|
|
],
|
|
[
|
|
0.1111111111111111,
|
|
"#46039f"
|
|
],
|
|
[
|
|
0.2222222222222222,
|
|
"#7201a8"
|
|
],
|
|
[
|
|
0.3333333333333333,
|
|
"#9c179e"
|
|
],
|
|
[
|
|
0.4444444444444444,
|
|
"#bd3786"
|
|
],
|
|
[
|
|
0.5555555555555556,
|
|
"#d8576b"
|
|
],
|
|
[
|
|
0.6666666666666666,
|
|
"#ed7953"
|
|
],
|
|
[
|
|
0.7777777777777778,
|
|
"#fb9f3a"
|
|
],
|
|
[
|
|
0.8888888888888888,
|
|
"#fdca26"
|
|
],
|
|
[
|
|
1,
|
|
"#f0f921"
|
|
]
|
|
],
|
|
"type": "heatmap"
|
|
}
|
|
],
|
|
"heatmapgl": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"colorscale": [
|
|
[
|
|
0,
|
|
"#0d0887"
|
|
],
|
|
[
|
|
0.1111111111111111,
|
|
"#46039f"
|
|
],
|
|
[
|
|
0.2222222222222222,
|
|
"#7201a8"
|
|
],
|
|
[
|
|
0.3333333333333333,
|
|
"#9c179e"
|
|
],
|
|
[
|
|
0.4444444444444444,
|
|
"#bd3786"
|
|
],
|
|
[
|
|
0.5555555555555556,
|
|
"#d8576b"
|
|
],
|
|
[
|
|
0.6666666666666666,
|
|
"#ed7953"
|
|
],
|
|
[
|
|
0.7777777777777778,
|
|
"#fb9f3a"
|
|
],
|
|
[
|
|
0.8888888888888888,
|
|
"#fdca26"
|
|
],
|
|
[
|
|
1,
|
|
"#f0f921"
|
|
]
|
|
],
|
|
"type": "heatmapgl"
|
|
}
|
|
],
|
|
"histogram": [
|
|
{
|
|
"marker": {
|
|
"pattern": {
|
|
"fillmode": "overlay",
|
|
"size": 10,
|
|
"solidity": 0.2
|
|
}
|
|
},
|
|
"type": "histogram"
|
|
}
|
|
],
|
|
"histogram2d": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"colorscale": [
|
|
[
|
|
0,
|
|
"#0d0887"
|
|
],
|
|
[
|
|
0.1111111111111111,
|
|
"#46039f"
|
|
],
|
|
[
|
|
0.2222222222222222,
|
|
"#7201a8"
|
|
],
|
|
[
|
|
0.3333333333333333,
|
|
"#9c179e"
|
|
],
|
|
[
|
|
0.4444444444444444,
|
|
"#bd3786"
|
|
],
|
|
[
|
|
0.5555555555555556,
|
|
"#d8576b"
|
|
],
|
|
[
|
|
0.6666666666666666,
|
|
"#ed7953"
|
|
],
|
|
[
|
|
0.7777777777777778,
|
|
"#fb9f3a"
|
|
],
|
|
[
|
|
0.8888888888888888,
|
|
"#fdca26"
|
|
],
|
|
[
|
|
1,
|
|
"#f0f921"
|
|
]
|
|
],
|
|
"type": "histogram2d"
|
|
}
|
|
],
|
|
"histogram2dcontour": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"colorscale": [
|
|
[
|
|
0,
|
|
"#0d0887"
|
|
],
|
|
[
|
|
0.1111111111111111,
|
|
"#46039f"
|
|
],
|
|
[
|
|
0.2222222222222222,
|
|
"#7201a8"
|
|
],
|
|
[
|
|
0.3333333333333333,
|
|
"#9c179e"
|
|
],
|
|
[
|
|
0.4444444444444444,
|
|
"#bd3786"
|
|
],
|
|
[
|
|
0.5555555555555556,
|
|
"#d8576b"
|
|
],
|
|
[
|
|
0.6666666666666666,
|
|
"#ed7953"
|
|
],
|
|
[
|
|
0.7777777777777778,
|
|
"#fb9f3a"
|
|
],
|
|
[
|
|
0.8888888888888888,
|
|
"#fdca26"
|
|
],
|
|
[
|
|
1,
|
|
"#f0f921"
|
|
]
|
|
],
|
|
"type": "histogram2dcontour"
|
|
}
|
|
],
|
|
"mesh3d": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"type": "mesh3d"
|
|
}
|
|
],
|
|
"parcoords": [
|
|
{
|
|
"line": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "parcoords"
|
|
}
|
|
],
|
|
"pie": [
|
|
{
|
|
"automargin": true,
|
|
"type": "pie"
|
|
}
|
|
],
|
|
"scatter": [
|
|
{
|
|
"fillpattern": {
|
|
"fillmode": "overlay",
|
|
"size": 10,
|
|
"solidity": 0.2
|
|
},
|
|
"type": "scatter"
|
|
}
|
|
],
|
|
"scatter3d": [
|
|
{
|
|
"line": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"marker": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "scatter3d"
|
|
}
|
|
],
|
|
"scattercarpet": [
|
|
{
|
|
"marker": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "scattercarpet"
|
|
}
|
|
],
|
|
"scattergeo": [
|
|
{
|
|
"marker": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "scattergeo"
|
|
}
|
|
],
|
|
"scattergl": [
|
|
{
|
|
"marker": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "scattergl"
|
|
}
|
|
],
|
|
"scattermapbox": [
|
|
{
|
|
"marker": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "scattermapbox"
|
|
}
|
|
],
|
|
"scatterpolar": [
|
|
{
|
|
"marker": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "scatterpolar"
|
|
}
|
|
],
|
|
"scatterpolargl": [
|
|
{
|
|
"marker": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "scatterpolargl"
|
|
}
|
|
],
|
|
"scatterternary": [
|
|
{
|
|
"marker": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"type": "scatterternary"
|
|
}
|
|
],
|
|
"surface": [
|
|
{
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
},
|
|
"colorscale": [
|
|
[
|
|
0,
|
|
"#0d0887"
|
|
],
|
|
[
|
|
0.1111111111111111,
|
|
"#46039f"
|
|
],
|
|
[
|
|
0.2222222222222222,
|
|
"#7201a8"
|
|
],
|
|
[
|
|
0.3333333333333333,
|
|
"#9c179e"
|
|
],
|
|
[
|
|
0.4444444444444444,
|
|
"#bd3786"
|
|
],
|
|
[
|
|
0.5555555555555556,
|
|
"#d8576b"
|
|
],
|
|
[
|
|
0.6666666666666666,
|
|
"#ed7953"
|
|
],
|
|
[
|
|
0.7777777777777778,
|
|
"#fb9f3a"
|
|
],
|
|
[
|
|
0.8888888888888888,
|
|
"#fdca26"
|
|
],
|
|
[
|
|
1,
|
|
"#f0f921"
|
|
]
|
|
],
|
|
"type": "surface"
|
|
}
|
|
],
|
|
"table": [
|
|
{
|
|
"cells": {
|
|
"fill": {
|
|
"color": "#EBF0F8"
|
|
},
|
|
"line": {
|
|
"color": "white"
|
|
}
|
|
},
|
|
"header": {
|
|
"fill": {
|
|
"color": "#C8D4E3"
|
|
},
|
|
"line": {
|
|
"color": "white"
|
|
}
|
|
},
|
|
"type": "table"
|
|
}
|
|
]
|
|
},
|
|
"layout": {
|
|
"annotationdefaults": {
|
|
"arrowcolor": "#2a3f5f",
|
|
"arrowhead": 0,
|
|
"arrowwidth": 1
|
|
},
|
|
"autotypenumbers": "strict",
|
|
"coloraxis": {
|
|
"colorbar": {
|
|
"outlinewidth": 0,
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"colorscale": {
|
|
"diverging": [
|
|
[
|
|
0,
|
|
"#8e0152"
|
|
],
|
|
[
|
|
0.1,
|
|
"#c51b7d"
|
|
],
|
|
[
|
|
0.2,
|
|
"#de77ae"
|
|
],
|
|
[
|
|
0.3,
|
|
"#f1b6da"
|
|
],
|
|
[
|
|
0.4,
|
|
"#fde0ef"
|
|
],
|
|
[
|
|
0.5,
|
|
"#f7f7f7"
|
|
],
|
|
[
|
|
0.6,
|
|
"#e6f5d0"
|
|
],
|
|
[
|
|
0.7,
|
|
"#b8e186"
|
|
],
|
|
[
|
|
0.8,
|
|
"#7fbc41"
|
|
],
|
|
[
|
|
0.9,
|
|
"#4d9221"
|
|
],
|
|
[
|
|
1,
|
|
"#276419"
|
|
]
|
|
],
|
|
"sequential": [
|
|
[
|
|
0,
|
|
"#0d0887"
|
|
],
|
|
[
|
|
0.1111111111111111,
|
|
"#46039f"
|
|
],
|
|
[
|
|
0.2222222222222222,
|
|
"#7201a8"
|
|
],
|
|
[
|
|
0.3333333333333333,
|
|
"#9c179e"
|
|
],
|
|
[
|
|
0.4444444444444444,
|
|
"#bd3786"
|
|
],
|
|
[
|
|
0.5555555555555556,
|
|
"#d8576b"
|
|
],
|
|
[
|
|
0.6666666666666666,
|
|
"#ed7953"
|
|
],
|
|
[
|
|
0.7777777777777778,
|
|
"#fb9f3a"
|
|
],
|
|
[
|
|
0.8888888888888888,
|
|
"#fdca26"
|
|
],
|
|
[
|
|
1,
|
|
"#f0f921"
|
|
]
|
|
],
|
|
"sequentialminus": [
|
|
[
|
|
0,
|
|
"#0d0887"
|
|
],
|
|
[
|
|
0.1111111111111111,
|
|
"#46039f"
|
|
],
|
|
[
|
|
0.2222222222222222,
|
|
"#7201a8"
|
|
],
|
|
[
|
|
0.3333333333333333,
|
|
"#9c179e"
|
|
],
|
|
[
|
|
0.4444444444444444,
|
|
"#bd3786"
|
|
],
|
|
[
|
|
0.5555555555555556,
|
|
"#d8576b"
|
|
],
|
|
[
|
|
0.6666666666666666,
|
|
"#ed7953"
|
|
],
|
|
[
|
|
0.7777777777777778,
|
|
"#fb9f3a"
|
|
],
|
|
[
|
|
0.8888888888888888,
|
|
"#fdca26"
|
|
],
|
|
[
|
|
1,
|
|
"#f0f921"
|
|
]
|
|
]
|
|
},
|
|
"colorway": [
|
|
"#636efa",
|
|
"#EF553B",
|
|
"#00cc96",
|
|
"#ab63fa",
|
|
"#FFA15A",
|
|
"#19d3f3",
|
|
"#FF6692",
|
|
"#B6E880",
|
|
"#FF97FF",
|
|
"#FECB52"
|
|
],
|
|
"font": {
|
|
"color": "#2a3f5f"
|
|
},
|
|
"geo": {
|
|
"bgcolor": "white",
|
|
"lakecolor": "white",
|
|
"landcolor": "#E5ECF6",
|
|
"showlakes": true,
|
|
"showland": true,
|
|
"subunitcolor": "white"
|
|
},
|
|
"hoverlabel": {
|
|
"align": "left"
|
|
},
|
|
"hovermode": "closest",
|
|
"mapbox": {
|
|
"style": "light"
|
|
},
|
|
"paper_bgcolor": "white",
|
|
"plot_bgcolor": "#E5ECF6",
|
|
"polar": {
|
|
"angularaxis": {
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"ticks": ""
|
|
},
|
|
"bgcolor": "#E5ECF6",
|
|
"radialaxis": {
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"scene": {
|
|
"xaxis": {
|
|
"backgroundcolor": "#E5ECF6",
|
|
"gridcolor": "white",
|
|
"gridwidth": 2,
|
|
"linecolor": "white",
|
|
"showbackground": true,
|
|
"ticks": "",
|
|
"zerolinecolor": "white"
|
|
},
|
|
"yaxis": {
|
|
"backgroundcolor": "#E5ECF6",
|
|
"gridcolor": "white",
|
|
"gridwidth": 2,
|
|
"linecolor": "white",
|
|
"showbackground": true,
|
|
"ticks": "",
|
|
"zerolinecolor": "white"
|
|
},
|
|
"zaxis": {
|
|
"backgroundcolor": "#E5ECF6",
|
|
"gridcolor": "white",
|
|
"gridwidth": 2,
|
|
"linecolor": "white",
|
|
"showbackground": true,
|
|
"ticks": "",
|
|
"zerolinecolor": "white"
|
|
}
|
|
},
|
|
"shapedefaults": {
|
|
"line": {
|
|
"color": "#2a3f5f"
|
|
}
|
|
},
|
|
"ternary": {
|
|
"aaxis": {
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"ticks": ""
|
|
},
|
|
"baxis": {
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"ticks": ""
|
|
},
|
|
"bgcolor": "#E5ECF6",
|
|
"caxis": {
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"ticks": ""
|
|
}
|
|
},
|
|
"title": {
|
|
"x": 0.05
|
|
},
|
|
"xaxis": {
|
|
"automargin": true,
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"ticks": "",
|
|
"title": {
|
|
"standoff": 15
|
|
},
|
|
"zerolinecolor": "white",
|
|
"zerolinewidth": 2
|
|
},
|
|
"yaxis": {
|
|
"automargin": true,
|
|
"gridcolor": "white",
|
|
"linecolor": "white",
|
|
"ticks": "",
|
|
"title": {
|
|
"standoff": 15
|
|
},
|
|
"zerolinecolor": "white",
|
|
"zerolinewidth": 2
|
|
}
|
|
}
|
|
},
|
|
"width": 1200,
|
|
"xaxis": {
|
|
"anchor": "y",
|
|
"domain": [
|
|
0,
|
|
1
|
|
],
|
|
"title": {
|
|
"text": "domain"
|
|
}
|
|
},
|
|
"yaxis": {
|
|
"anchor": "x",
|
|
"domain": [
|
|
0,
|
|
1
|
|
],
|
|
"title": {
|
|
"text": "count"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import pandas as pd\n",
|
|
"from collections import Counter\n",
|
|
"import plotly.express as px\n",
|
|
"\n",
|
|
"df = pd.DataFrame(Counter(y).most_common(), columns=[\"domain\", \"count\"])\n",
|
|
"px.bar(df, \"domain\", \"count\", width=1200, height=400).show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Optimise and train Multinomial Naive Bayes classifier"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from sklearn.naive_bayes import MultinomialNB\n",
|
|
"from sklearn.pipeline import Pipeline\n",
|
|
"from sklearn.feature_extraction.text import TfidfVectorizer\n",
|
|
"\n",
|
|
"\n",
|
|
"def create_pipeline() -> Pipeline:\n",
|
|
" return Pipeline(\n",
|
|
" steps=[\n",
|
|
" (\"vectorizer\", TfidfVectorizer(sublinear_tf=True)),\n",
|
|
" (\"classifier\", MultinomialNB()),\n",
|
|
" ]\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Fitting 3 folds for each of 24 candidates, totalling 72 fits\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>mean_fit_time</th>\n",
|
|
" <th>std_fit_time</th>\n",
|
|
" <th>mean_score_time</th>\n",
|
|
" <th>std_score_time</th>\n",
|
|
" <th>param_classifier__alpha</th>\n",
|
|
" <th>param_classifier__fit_prior</th>\n",
|
|
" <th>param_vectorizer__max_df</th>\n",
|
|
" <th>param_vectorizer__min_df</th>\n",
|
|
" <th>params</th>\n",
|
|
" <th>split0_test_score</th>\n",
|
|
" <th>split1_test_score</th>\n",
|
|
" <th>split2_test_score</th>\n",
|
|
" <th>mean_test_score</th>\n",
|
|
" <th>std_test_score</th>\n",
|
|
" <th>rank_test_score</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>7</th>\n",
|
|
" <td>1.962260</td>\n",
|
|
" <td>0.147449</td>\n",
|
|
" <td>0.935357</td>\n",
|
|
" <td>0.063659</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>20</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.485030</td>\n",
|
|
" <td>0.463849</td>\n",
|
|
" <td>0.481840</td>\n",
|
|
" <td>0.476906</td>\n",
|
|
" <td>0.009324</td>\n",
|
|
" <td>1</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>10</th>\n",
|
|
" <td>1.942605</td>\n",
|
|
" <td>0.111027</td>\n",
|
|
" <td>0.952361</td>\n",
|
|
" <td>0.066812</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>20</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.482890</td>\n",
|
|
" <td>0.459556</td>\n",
|
|
" <td>0.479362</td>\n",
|
|
" <td>0.473936</td>\n",
|
|
" <td>0.010270</td>\n",
|
|
" <td>2</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>19</th>\n",
|
|
" <td>2.145152</td>\n",
|
|
" <td>0.068978</td>\n",
|
|
" <td>1.002291</td>\n",
|
|
" <td>0.047358</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>20</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.467330</td>\n",
|
|
" <td>0.442994</td>\n",
|
|
" <td>0.464302</td>\n",
|
|
" <td>0.458208</td>\n",
|
|
" <td>0.010829</td>\n",
|
|
" <td>3</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>22</th>\n",
|
|
" <td>1.971888</td>\n",
|
|
" <td>0.126950</td>\n",
|
|
" <td>0.739795</td>\n",
|
|
" <td>0.071551</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>20</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.454830</td>\n",
|
|
" <td>0.422902</td>\n",
|
|
" <td>0.450677</td>\n",
|
|
" <td>0.442803</td>\n",
|
|
" <td>0.014174</td>\n",
|
|
" <td>4</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>6</th>\n",
|
|
" <td>1.861275</td>\n",
|
|
" <td>0.013389</td>\n",
|
|
" <td>1.058907</td>\n",
|
|
" <td>0.111122</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>5</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.456127</td>\n",
|
|
" <td>0.422456</td>\n",
|
|
" <td>0.443827</td>\n",
|
|
" <td>0.440803</td>\n",
|
|
" <td>0.013912</td>\n",
|
|
" <td>5</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>11</th>\n",
|
|
" <td>1.825397</td>\n",
|
|
" <td>0.105754</td>\n",
|
|
" <td>0.892227</td>\n",
|
|
" <td>0.057003</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>100</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.438232</td>\n",
|
|
" <td>0.440464</td>\n",
|
|
" <td>0.422667</td>\n",
|
|
" <td>0.433788</td>\n",
|
|
" <td>0.007916</td>\n",
|
|
" <td>6</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>23</th>\n",
|
|
" <td>1.693333</td>\n",
|
|
" <td>0.009667</td>\n",
|
|
" <td>0.501491</td>\n",
|
|
" <td>0.006545</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>100</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.433915</td>\n",
|
|
" <td>0.439470</td>\n",
|
|
" <td>0.416031</td>\n",
|
|
" <td>0.429805</td>\n",
|
|
" <td>0.010001</td>\n",
|
|
" <td>7</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>8</th>\n",
|
|
" <td>2.008045</td>\n",
|
|
" <td>0.145330</td>\n",
|
|
" <td>0.944559</td>\n",
|
|
" <td>0.155925</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>100</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.436178</td>\n",
|
|
" <td>0.425724</td>\n",
|
|
" <td>0.418396</td>\n",
|
|
" <td>0.426766</td>\n",
|
|
" <td>0.007297</td>\n",
|
|
" <td>8</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>20</th>\n",
|
|
" <td>1.749200</td>\n",
|
|
" <td>0.022959</td>\n",
|
|
" <td>0.889532</td>\n",
|
|
" <td>0.047517</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>100</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.428215</td>\n",
|
|
" <td>0.425398</td>\n",
|
|
" <td>0.411051</td>\n",
|
|
" <td>0.421555</td>\n",
|
|
" <td>0.007516</td>\n",
|
|
" <td>9</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>9</th>\n",
|
|
" <td>1.960889</td>\n",
|
|
" <td>0.098004</td>\n",
|
|
" <td>0.985957</td>\n",
|
|
" <td>0.080925</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>5</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.430638</td>\n",
|
|
" <td>0.406619</td>\n",
|
|
" <td>0.420213</td>\n",
|
|
" <td>0.419157</td>\n",
|
|
" <td>0.009834</td>\n",
|
|
" <td>10</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>18</th>\n",
|
|
" <td>1.807799</td>\n",
|
|
" <td>0.064891</td>\n",
|
|
" <td>0.881872</td>\n",
|
|
" <td>0.030810</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>5</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.402402</td>\n",
|
|
" <td>0.372353</td>\n",
|
|
" <td>0.386189</td>\n",
|
|
" <td>0.386981</td>\n",
|
|
" <td>0.012280</td>\n",
|
|
" <td>11</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>2.009232</td>\n",
|
|
" <td>0.043125</td>\n",
|
|
" <td>0.899676</td>\n",
|
|
" <td>0.036977</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>20</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.389797</td>\n",
|
|
" <td>0.372619</td>\n",
|
|
" <td>0.388358</td>\n",
|
|
" <td>0.383591</td>\n",
|
|
" <td>0.007781</td>\n",
|
|
" <td>12</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>1.868087</td>\n",
|
|
" <td>0.094739</td>\n",
|
|
" <td>1.005353</td>\n",
|
|
" <td>0.101466</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>20</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.379323</td>\n",
|
|
" <td>0.364667</td>\n",
|
|
" <td>0.379060</td>\n",
|
|
" <td>0.374350</td>\n",
|
|
" <td>0.006848</td>\n",
|
|
" <td>13</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>21</th>\n",
|
|
" <td>1.958430</td>\n",
|
|
" <td>0.039639</td>\n",
|
|
" <td>0.890963</td>\n",
|
|
" <td>0.012546</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>False</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>5</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.366936</td>\n",
|
|
" <td>0.343361</td>\n",
|
|
" <td>0.363883</td>\n",
|
|
" <td>0.358060</td>\n",
|
|
" <td>0.010468</td>\n",
|
|
" <td>14</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>5</th>\n",
|
|
" <td>1.940692</td>\n",
|
|
" <td>0.018320</td>\n",
|
|
" <td>0.898865</td>\n",
|
|
" <td>0.030651</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>100</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.354850</td>\n",
|
|
" <td>0.349237</td>\n",
|
|
" <td>0.342737</td>\n",
|
|
" <td>0.348941</td>\n",
|
|
" <td>0.004950</td>\n",
|
|
" <td>15</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>1.855691</td>\n",
|
|
" <td>0.029506</td>\n",
|
|
" <td>0.866492</td>\n",
|
|
" <td>0.038048</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>100</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.349005</td>\n",
|
|
" <td>0.341832</td>\n",
|
|
" <td>0.328617</td>\n",
|
|
" <td>0.339818</td>\n",
|
|
" <td>0.008444</td>\n",
|
|
" <td>16</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>17</th>\n",
|
|
" <td>1.798559</td>\n",
|
|
" <td>0.103497</td>\n",
|
|
" <td>0.888273</td>\n",
|
|
" <td>0.069050</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>100</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.312332</td>\n",
|
|
" <td>0.297655</td>\n",
|
|
" <td>0.307471</td>\n",
|
|
" <td>0.305819</td>\n",
|
|
" <td>0.006104</td>\n",
|
|
" <td>17</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>14</th>\n",
|
|
" <td>2.016041</td>\n",
|
|
" <td>0.232138</td>\n",
|
|
" <td>0.967630</td>\n",
|
|
" <td>0.146144</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>100</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.304942</td>\n",
|
|
" <td>0.296921</td>\n",
|
|
" <td>0.297121</td>\n",
|
|
" <td>0.299661</td>\n",
|
|
" <td>0.003735</td>\n",
|
|
" <td>18</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>13</th>\n",
|
|
" <td>1.829513</td>\n",
|
|
" <td>0.112645</td>\n",
|
|
" <td>0.885848</td>\n",
|
|
" <td>0.027726</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>20</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.301539</td>\n",
|
|
" <td>0.285396</td>\n",
|
|
" <td>0.297272</td>\n",
|
|
" <td>0.294736</td>\n",
|
|
" <td>0.006830</td>\n",
|
|
" <td>19</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>1.905362</td>\n",
|
|
" <td>0.018052</td>\n",
|
|
" <td>0.885552</td>\n",
|
|
" <td>0.023985</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>5</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.295635</td>\n",
|
|
" <td>0.276759</td>\n",
|
|
" <td>0.296270</td>\n",
|
|
" <td>0.289555</td>\n",
|
|
" <td>0.009052</td>\n",
|
|
" <td>20</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>16</th>\n",
|
|
" <td>1.793688</td>\n",
|
|
" <td>0.049995</td>\n",
|
|
" <td>0.921301</td>\n",
|
|
" <td>0.060980</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>20</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.286746</td>\n",
|
|
" <td>0.272260</td>\n",
|
|
" <td>0.277084</td>\n",
|
|
" <td>0.278696</td>\n",
|
|
" <td>0.006023</td>\n",
|
|
" <td>21</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>2.078568</td>\n",
|
|
" <td>0.045549</td>\n",
|
|
" <td>0.963691</td>\n",
|
|
" <td>0.048281</td>\n",
|
|
" <td>0.5</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>5</td>\n",
|
|
" <td>{'classifier__alpha': 0.5, 'classifier__fit_pr...</td>\n",
|
|
" <td>0.276452</td>\n",
|
|
" <td>0.265509</td>\n",
|
|
" <td>0.268949</td>\n",
|
|
" <td>0.270303</td>\n",
|
|
" <td>0.004569</td>\n",
|
|
" <td>22</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>12</th>\n",
|
|
" <td>1.839506</td>\n",
|
|
" <td>0.048910</td>\n",
|
|
" <td>0.921812</td>\n",
|
|
" <td>0.010374</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.05</td>\n",
|
|
" <td>5</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.183196</td>\n",
|
|
" <td>0.186144</td>\n",
|
|
" <td>0.180323</td>\n",
|
|
" <td>0.183221</td>\n",
|
|
" <td>0.002376</td>\n",
|
|
" <td>23</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>15</th>\n",
|
|
" <td>1.909279</td>\n",
|
|
" <td>0.110639</td>\n",
|
|
" <td>1.056087</td>\n",
|
|
" <td>0.129738</td>\n",
|
|
" <td>1</td>\n",
|
|
" <td>True</td>\n",
|
|
" <td>0.1</td>\n",
|
|
" <td>5</td>\n",
|
|
" <td>{'classifier__alpha': 1, 'classifier__fit_prio...</td>\n",
|
|
" <td>0.165277</td>\n",
|
|
" <td>0.165840</td>\n",
|
|
" <td>0.167088</td>\n",
|
|
" <td>0.166068</td>\n",
|
|
" <td>0.000757</td>\n",
|
|
" <td>24</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>"
|
|
],
|
|
"text/plain": [
|
|
" mean_fit_time std_fit_time mean_score_time std_score_time \\\n",
|
|
"7 1.962260 0.147449 0.935357 0.063659 \n",
|
|
"10 1.942605 0.111027 0.952361 0.066812 \n",
|
|
"19 2.145152 0.068978 1.002291 0.047358 \n",
|
|
"22 1.971888 0.126950 0.739795 0.071551 \n",
|
|
"6 1.861275 0.013389 1.058907 0.111122 \n",
|
|
"11 1.825397 0.105754 0.892227 0.057003 \n",
|
|
"23 1.693333 0.009667 0.501491 0.006545 \n",
|
|
"8 2.008045 0.145330 0.944559 0.155925 \n",
|
|
"20 1.749200 0.022959 0.889532 0.047517 \n",
|
|
"9 1.960889 0.098004 0.985957 0.080925 \n",
|
|
"18 1.807799 0.064891 0.881872 0.030810 \n",
|
|
"1 2.009232 0.043125 0.899676 0.036977 \n",
|
|
"4 1.868087 0.094739 1.005353 0.101466 \n",
|
|
"21 1.958430 0.039639 0.890963 0.012546 \n",
|
|
"5 1.940692 0.018320 0.898865 0.030651 \n",
|
|
"2 1.855691 0.029506 0.866492 0.038048 \n",
|
|
"17 1.798559 0.103497 0.888273 0.069050 \n",
|
|
"14 2.016041 0.232138 0.967630 0.146144 \n",
|
|
"13 1.829513 0.112645 0.885848 0.027726 \n",
|
|
"0 1.905362 0.018052 0.885552 0.023985 \n",
|
|
"16 1.793688 0.049995 0.921301 0.060980 \n",
|
|
"3 2.078568 0.045549 0.963691 0.048281 \n",
|
|
"12 1.839506 0.048910 0.921812 0.010374 \n",
|
|
"15 1.909279 0.110639 1.056087 0.129738 \n",
|
|
"\n",
|
|
" param_classifier__alpha param_classifier__fit_prior \\\n",
|
|
"7 0.5 False \n",
|
|
"10 0.5 False \n",
|
|
"19 1 False \n",
|
|
"22 1 False \n",
|
|
"6 0.5 False \n",
|
|
"11 0.5 False \n",
|
|
"23 1 False \n",
|
|
"8 0.5 False \n",
|
|
"20 1 False \n",
|
|
"9 0.5 False \n",
|
|
"18 1 False \n",
|
|
"1 0.5 True \n",
|
|
"4 0.5 True \n",
|
|
"21 1 False \n",
|
|
"5 0.5 True \n",
|
|
"2 0.5 True \n",
|
|
"17 1 True \n",
|
|
"14 1 True \n",
|
|
"13 1 True \n",
|
|
"0 0.5 True \n",
|
|
"16 1 True \n",
|
|
"3 0.5 True \n",
|
|
"12 1 True \n",
|
|
"15 1 True \n",
|
|
"\n",
|
|
" param_vectorizer__max_df param_vectorizer__min_df \\\n",
|
|
"7 0.05 20 \n",
|
|
"10 0.1 20 \n",
|
|
"19 0.05 20 \n",
|
|
"22 0.1 20 \n",
|
|
"6 0.05 5 \n",
|
|
"11 0.1 100 \n",
|
|
"23 0.1 100 \n",
|
|
"8 0.05 100 \n",
|
|
"20 0.05 100 \n",
|
|
"9 0.1 5 \n",
|
|
"18 0.05 5 \n",
|
|
"1 0.05 20 \n",
|
|
"4 0.1 20 \n",
|
|
"21 0.1 5 \n",
|
|
"5 0.1 100 \n",
|
|
"2 0.05 100 \n",
|
|
"17 0.1 100 \n",
|
|
"14 0.05 100 \n",
|
|
"13 0.05 20 \n",
|
|
"0 0.05 5 \n",
|
|
"16 0.1 20 \n",
|
|
"3 0.1 5 \n",
|
|
"12 0.05 5 \n",
|
|
"15 0.1 5 \n",
|
|
"\n",
|
|
" params split0_test_score \\\n",
|
|
"7 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.485030 \n",
|
|
"10 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.482890 \n",
|
|
"19 {'classifier__alpha': 1, 'classifier__fit_prio... 0.467330 \n",
|
|
"22 {'classifier__alpha': 1, 'classifier__fit_prio... 0.454830 \n",
|
|
"6 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.456127 \n",
|
|
"11 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.438232 \n",
|
|
"23 {'classifier__alpha': 1, 'classifier__fit_prio... 0.433915 \n",
|
|
"8 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.436178 \n",
|
|
"20 {'classifier__alpha': 1, 'classifier__fit_prio... 0.428215 \n",
|
|
"9 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.430638 \n",
|
|
"18 {'classifier__alpha': 1, 'classifier__fit_prio... 0.402402 \n",
|
|
"1 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.389797 \n",
|
|
"4 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.379323 \n",
|
|
"21 {'classifier__alpha': 1, 'classifier__fit_prio... 0.366936 \n",
|
|
"5 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.354850 \n",
|
|
"2 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.349005 \n",
|
|
"17 {'classifier__alpha': 1, 'classifier__fit_prio... 0.312332 \n",
|
|
"14 {'classifier__alpha': 1, 'classifier__fit_prio... 0.304942 \n",
|
|
"13 {'classifier__alpha': 1, 'classifier__fit_prio... 0.301539 \n",
|
|
"0 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.295635 \n",
|
|
"16 {'classifier__alpha': 1, 'classifier__fit_prio... 0.286746 \n",
|
|
"3 {'classifier__alpha': 0.5, 'classifier__fit_pr... 0.276452 \n",
|
|
"12 {'classifier__alpha': 1, 'classifier__fit_prio... 0.183196 \n",
|
|
"15 {'classifier__alpha': 1, 'classifier__fit_prio... 0.165277 \n",
|
|
"\n",
|
|
" split1_test_score split2_test_score mean_test_score std_test_score \\\n",
|
|
"7 0.463849 0.481840 0.476906 0.009324 \n",
|
|
"10 0.459556 0.479362 0.473936 0.010270 \n",
|
|
"19 0.442994 0.464302 0.458208 0.010829 \n",
|
|
"22 0.422902 0.450677 0.442803 0.014174 \n",
|
|
"6 0.422456 0.443827 0.440803 0.013912 \n",
|
|
"11 0.440464 0.422667 0.433788 0.007916 \n",
|
|
"23 0.439470 0.416031 0.429805 0.010001 \n",
|
|
"8 0.425724 0.418396 0.426766 0.007297 \n",
|
|
"20 0.425398 0.411051 0.421555 0.007516 \n",
|
|
"9 0.406619 0.420213 0.419157 0.009834 \n",
|
|
"18 0.372353 0.386189 0.386981 0.012280 \n",
|
|
"1 0.372619 0.388358 0.383591 0.007781 \n",
|
|
"4 0.364667 0.379060 0.374350 0.006848 \n",
|
|
"21 0.343361 0.363883 0.358060 0.010468 \n",
|
|
"5 0.349237 0.342737 0.348941 0.004950 \n",
|
|
"2 0.341832 0.328617 0.339818 0.008444 \n",
|
|
"17 0.297655 0.307471 0.305819 0.006104 \n",
|
|
"14 0.296921 0.297121 0.299661 0.003735 \n",
|
|
"13 0.285396 0.297272 0.294736 0.006830 \n",
|
|
"0 0.276759 0.296270 0.289555 0.009052 \n",
|
|
"16 0.272260 0.277084 0.278696 0.006023 \n",
|
|
"3 0.265509 0.268949 0.270303 0.004569 \n",
|
|
"12 0.186144 0.180323 0.183221 0.002376 \n",
|
|
"15 0.165840 0.167088 0.166068 0.000757 \n",
|
|
"\n",
|
|
" rank_test_score \n",
|
|
"7 1 \n",
|
|
"10 2 \n",
|
|
"19 3 \n",
|
|
"22 4 \n",
|
|
"6 5 \n",
|
|
"11 6 \n",
|
|
"23 7 \n",
|
|
"8 8 \n",
|
|
"20 9 \n",
|
|
"9 10 \n",
|
|
"18 11 \n",
|
|
"1 12 \n",
|
|
"4 13 \n",
|
|
"21 14 \n",
|
|
"5 15 \n",
|
|
"2 16 \n",
|
|
"17 17 \n",
|
|
"14 18 \n",
|
|
"13 19 \n",
|
|
"0 20 \n",
|
|
"16 21 \n",
|
|
"3 22 \n",
|
|
"12 23 \n",
|
|
"15 24 "
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from sklearn.model_selection import GridSearchCV\n",
|
|
"\n",
|
|
"optimisation_pipeline = GridSearchCV(\n",
|
|
" create_pipeline(),\n",
|
|
" {\n",
|
|
" \"vectorizer__min_df\": [5, 20, 100],\n",
|
|
" \"vectorizer__max_df\": [0.05, 0.1],\n",
|
|
" \"classifier__alpha\": [0.5, 1],\n",
|
|
" \"classifier__fit_prior\": [True, False],\n",
|
|
" },\n",
|
|
" scoring=\"f1_macro\",\n",
|
|
" cv=3,\n",
|
|
" n_jobs=-1,\n",
|
|
" verbose=1,\n",
|
|
")\n",
|
|
"optimisation_pipeline.fit(X, y)\n",
|
|
"\n",
|
|
"results = pd.DataFrame(optimisation_pipeline.cv_results_)\n",
|
|
"results.sort_values(\"rank_test_score\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[('vectorizer',\n",
|
|
" TfidfVectorizer(max_df=0.05, min_df=20, sublinear_tf=True)),\n",
|
|
" ('classifier', MultinomialNB(alpha=0.5, fit_prior=False))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Pipeline</label><div class=\"sk-toggleable__content\"><pre>Pipeline(steps=[('vectorizer',\n",
|
|
" TfidfVectorizer(max_df=0.05, min_df=20, sublinear_tf=True)),\n",
|
|
" ('classifier', MultinomialNB(alpha=0.5, fit_prior=False))])</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">TfidfVectorizer</label><div class=\"sk-toggleable__content\"><pre>TfidfVectorizer(max_df=0.05, min_df=20, sublinear_tf=True)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MultinomialNB</label><div class=\"sk-toggleable__content\"><pre>MultinomialNB(alpha=0.5, fit_prior=False)</pre></div></div></div></div></div></div></div>"
|
|
],
|
|
"text/plain": [
|
|
"Pipeline(steps=[('vectorizer',\n",
|
|
" TfidfVectorizer(max_df=0.05, min_df=20, sublinear_tf=True)),\n",
|
|
" ('classifier', MultinomialNB(alpha=0.5, fit_prior=False))])"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from sklearn import set_config\n",
|
|
"\n",
|
|
"set_config(display=\"diagram\")\n",
|
|
"\n",
|
|
"classifier = create_pipeline()\n",
|
|
"classifier.set_params(**optimisation_pipeline.best_params_)\n",
|
|
"classifier.fit(X, y)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Export the model using GreatAI"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,592 | INFO | Copying file for small-domain-prediction-0\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,613 | INFO | Compressing small-domain-prediction-0\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,917 | INFO | Uploading /tmp/tmpvxez8op8/small-domain-prediction-0.tar.gz to Mongo (GridFS)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,972 | INFO | Uploading small-domain-prediction-0.tar.gz 0.26/1.85 MB (14.1%)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,974 | INFO | Uploading small-domain-prediction-0.tar.gz 0.52/1.85 MB (28.2%)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,975 | INFO | Uploading small-domain-prediction-0.tar.gz 0.78/1.85 MB (42.3%)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,977 | INFO | Uploading small-domain-prediction-0.tar.gz 1.04/1.85 MB (56.4%)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,979 | INFO | Uploading small-domain-prediction-0.tar.gz 1.31/1.85 MB (70.5%)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,980 | INFO | Uploading small-domain-prediction-0.tar.gz 1.57/1.85 MB (84.7%)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,982 | INFO | Uploading small-domain-prediction-0.tar.gz 1.83/1.85 MB (98.8%)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,982 | INFO | Uploading small-domain-prediction-0.tar.gz 1.85/1.85 MB (100.0%)\u001b[0m\n",
|
|
"\u001b[38;5;39m2022-06-25 14:50:53,985 | INFO | Model small-domain-prediction uploaded with version 0\u001b[0m\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"'small-domain-prediction:0'"
|
|
]
|
|
},
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from great_ai import save_model\n",
|
|
"\n",
|
|
"\n",
|
|
"save_model(classifier, key=\"small-domain-prediction\", keep_last_n=5)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Next: [Part 3](deploy.ipynb)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3.10.4 ('.env': venv)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.4"
|
|
},
|
|
"orig_nbformat": 4,
|
|
"vscode": {
|
|
"interpreter": {
|
|
"hash": "c1f394f9662881005685eeb18d8f9f77079b1b8b9a5ece1f825bfa01fcb7f52f"
|
|
}
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|