Simple PRAW script for scraping concatenated posts, both relevant and irrelevant
to one another, depth-first from random Reddit comment pools. To use, first set
the environment variables REDDIT_CLIENT_ID and REDDIT_CLIENT_SECRET, then
run like so:
$ python data.py --length 0.1G --encoding utf-8 -o annotations.txt
note: while it is possible to redirect stdout, using --opath allows
data.py to remember where it left off and account for progress accordingly.
data.py— print supervised fastText annotations to stdout
TODO: measure toxicity v. supportiveness: Some replies are constructive, and some aren't; we should be able to measure either a lack of attacks or presence of positive features in responses, though fastText may not represent the most accurate means to accomplish this