Change Details

* [ ] Blocked by subtasks to prepare the scraper * [ ] Choose a handful of wikis to test our scraper against. enwiki, dewiki, and a few more with unique Cite * [ ] Pull the first 10k lines for these wikis, without decompressing or transferring the entire tarballs. * [ ] Scrape each sample to a separate directory * [ ] Run aggregation on each sample * [ ] Post the results somewhere the whole team can review (TBD)

* [ ] Blocked by subtasks to prepare the scraper * [ ] Choose a handful of wikis to test our scraper against. enwiki, dewiki, and fawiki (because it's common to use <ref> without creating the tag via template) * [ ] Pull the first 10k lines for these wikis, without decompressing or transferring the entire tarballs. * [ ] Scrape each sample to a separate directory * [ ] Run aggregation on each sample * [ ] Post the results somewhere the whole team can review (TBD)

* [ ] Blocked by subtasks to prepare the scraper * [ ] Choose a handful of wikis to test our scraper against. enwiki, dewiki, and a few morefawiki (because it's common to use <ref> with unique Cite out creating the tag via template) * [ ] Pull the first 10k lines for these wikis, without decompressing or transferring the entire tarballs. * [ ] Scrape each sample to a separate directory * [ ] Run aggregation on each sample * [ ] Post the results somewhere the whole team can review (TBD)