As a way to measure the Metrics Platform's progress in achieve our goals, we determined a set of measurable success criteria:
- decrease the time to data by X%
- decrease the amount of engineering days required to create instruments by X%
We now to to establish tooling or processes to collect this information in an ongoing way as well as establish baselines
- There is a publicly-accessible list of data sources that we could use to establish baselines for Metrics Platform metrics
- Each list entry should include:
- Whether the data are public or private
- An example query or an example script to extract the required data
- Sense-check all of the above with @nettrom_WMF
Possible Data Sources
- Gerrit – How can we extract the timestamp of the first commit associated with an instrumentation task? Is there other/more information available?
- Phabricator – Can we use Phabricator comments to estimate how long it took to build an instrument? Is there other/more information available?
- Surveys – R&DS ran a survey about this in 2020-21 (see https://wikimedia.slack.com/archives/C01DFMX6QLB/p1704816837945919)