We need our //golden// data retriever codebase to be more modular and have better support for addition of new modules, which will be very helpful when we start doing ZRR calculation for "well-behaved searches" (see T150370 & T150901). Testing if a new script works and backfilling missing data is a huge pain right now.
I've worked with the package [[ https://cran.r-project.org/package=optparse | optparse ]] before and it would make it easy/possible to do things like:
```lang=bash
# See all options:
Rscript run.R -h
Rscript run.R --help
# Fetch latest numbers for all teams' metrics:
Rscript run.R
# Backfill a specific metric for a specific team on a particular date:
Rscript run.R --backfill --date=20161031 --team=search --metric=zrr
# Backfill a specific team's metrics for a range of dates:
Rscript run.R --backfill --date=range --from=20161001 --to=20161031 --team=wdqs
# Detect missing data:
Rscript run.R --detect --date=range --from=20161021 --to=20161105
# would return a table of missing data: | date | team | metric |
# Detect missing data and attempt to backfill:
Rscript run.R --detect --backfill --date=range --from=20161021 --to=20161105
# Show diagnostic messages and show/return data without writing any of it to file:
Rscript run.R --dryrun
Rscript run.R --team=portal --metric=pageviews --dryrun
```
**Note**: in some ways this is related to T145445, and would actually make that task a little bit easier.After talking with Analytics, e.g. `Rscript run.R --test`Chelsy and I decided to migrate our codebase to use their [[ https://wikitech.wikimedia.org/wiki/Analytics/Reportupdater | Reportupdater ]] infrastructure as it seems to meet our needs.