History

András Schmelczer d488627376 Add colours to logging Signed-off-by: András Schmelczer <andras@schmelczer.dev>		2022-04-10 09:41:04 +02:00
..
models	Format	2022-04-02 13:57:16 +02:00
.gitignore	More experimentation	2022-04-03 21:46:35 +02:00
config.py	Add Dash	2022-04-06 21:05:52 +02:00
main_batch.py	Add log_argument	2022-04-09 22:02:00 +02:00
main_service.py	Add log_argument	2022-04-09 22:02:00 +02:00
predict_domain.py	Add log_argument	2022-04-09 22:02:00 +02:00
preprocess.py	Format	2022-04-02 21:59:09 +02:00
README.md	More experimentation	2022-04-03 21:46:35 +02:00
train.ipynb	Add colours to logging	2022-04-10 09:41:04 +02:00

README.md

Train Domain classifier from the semantic scholar dataset

Upload the dataset (or a part of it) to shared infrastructure

mkdir ss-data && cd ss-data
wget https://s3-us-west-2.amazonaws.com/ai2-s2-research-public/open-corpus/2022-02-01/manifest.txt
wget -B https://s3-us-west-2.amazonaws.com/ai2-s2-research-public/open-corpus/2022-02-01/ -i manifest.txt
cd -
python3 -m good_ai.open_s3 --secrets s3.ini --push ss-data
rm -rf ss-data