This task involves the work of investigating the approaches WE 1.1 could consider pursuing to measure the KR's collective impact on the rate at which editors with ≤100 cumulative edits publish constructive edits on mobile web.

Approaches

1. At the end of the year, measure the relative increase in the rate of constructive edits across all Wikipedias compared to the mobile web baseline (71% as established in T389404). Note: This would depend on interventions being widely deployed by the time of analysis.
2. Calculate the proportion of interventions we deployed and evaluated through controlled experiments that met and/or exceeded the 4% constructive rate KR target.
3. Run a controlled experiment using a global holdout group. In this experiment, we would run an experiment at the end of the year comparing a group of users who have access to all the interventions (test) to a holdout group that never gain access to the interventions we are incrementally deploying and evaluating (control). The feasbility of this option is being investigated in T392959.

References

• Year-over-year approach (Nov. 2024): T375926#10300625
• Single controlled experiment: notes and Slack discussion
  • Related: T392959: [SPIKE] Investigate feasibility of creating a global "hold out" group for each WE 1.1 experiment