Thanks for making the task! I'm going to start with the exhaustive, MW API-based approach that I saw was mentioned in the IRC backscroll. Even in the crude form it's in now, I can pull 23k article titles for WikiProject Medicine in ten seconds or so, and do the sampling in memory. Another benefit of this approach is that we can get /somewhat/ stable samples by setting the random seed (note to self: and by ordering the results by page ID).

It seems to me that we want to have a long-term snapshot of input data. This is helpful when we discover some type of statistical anomaly, it'll be nice to check the *exact sample* that data originates from. So, if we use the API, then it'll be important that we keep a snapshot of the data in our repo.

I propose a simple guideline. For cases where a specific version of a dataset is stored long term with a unique identifier, it is OK to not store that dataset in our repositories. Instead, we'll include the unique identifier in our Makefiles/configuration.

For cases where a specific version of a dataset is not stored long term or cannot be exactly re-created using a unique identifier, we should commit the simplest version of the dataset to our repositories for re-use, but we should also maintain the provenance of the datset in our Makefiles/configuration.