1. T325713 makes it possible to detect whether an edit involves someone adding a reference
2. T333714 makes it possible to detect whether an edit involves someone adding new information
3. T342462 makes it possible to identify edits the reference check is shown within

...this task involves the work of combining/composing the three pieces of information above into a "view" that enables people across the Movement [i] to see how "1." and "3." relate.

Story

As someone who is motivated to evaluate the extent to which the reference Edit Check is having a net positive impact on a particular Wikipedia, I'd like an easy way to see aggregate counts of various editing metrics, so that I can decide what – if any – adjustments I think need to be made to who the reference Edit Check is activated for and how the check is configured.

Requirements

Meta

1. Data needs to be publicly available
2. Ideally data is presented in a way/in a place where it can be annotated so that we can collectively track of how certain changes [ii] may or may not cause detectable shifts in aggregate behavior
3. People need to be able to see:
   1. How the proportion of new content edits that include a reference varies between edits in which the reference Edit Check was and was not shown
   2. How "3.A." varies over time
   3. How the proportion of new content edits that are reverted varies between edits in which the reference Edit Check was and was not shown
   4. The sum of references Edit Check prompted people to add

References

• Using Abuse Filter "hits" over time as a proxy for how effective a given Edit Check is at reducing the prevalence of a particular kind of edit. //Via @Xaosflux: mw:Talk:Edit check

Approaches

• Approach #1: Grafana
  • Example: Growth team product KPIs
• Approach #2: TBD
• Approach #3: TBD

i. Read: interested volunteers and members of the Editing Team [for now]
ii. Where "certain changes" in this context could mean:

• Changes in the the population(s) within a project Edit Check is enabled for
• Changes in the percentage of a population within a project Edit Check is enabled for
• Changes to configuration variables that determine the thresholds that must be "met" for Edit Check to become activated