≥2 weeks after starting of the Reference Check A/B Test (T400101), we will check on a set of leading indicators (outlined below).
We will use this ticket to scope and conduct this analysis.
=== Analysis timing
**Analysis can begin on 18 November 2025**
NOTE: analysis will begin ≥2 weeks after starting of the Reference Check A/B Test
Decisions to be made
- [ ] 1. What – if any – **UX** adjustments/investigations will we prioritize for us to be confident moving forward with evaluating the Reference Check's impact in T400101?
- [ ] 2. What – if any – adjusts will we make to **experiment's design** to ensure enough newcomers are encountering Reference Check for us to draw statistically significant conclusions about it?
=== Leading indicators
//Metrics//
|ID|Name|Owner|Metric(s) for Evaluation | Conclusion
|---|---|---|---|---
|1.|Newcomers are not encountering Reference Check | Editing | Proportion of new content edits Reference Check is shown within|
|2.|Newcomers are not understanding the feature| Editing| Proportion of contributors that are presented Reference Check and abandon their edits|
|3.|People deem Reference Check irrelevant| Editing |Proportion of edits wherein people elect NOT to cite the text they are attempting to add|
|4.|Reference Check is causing disruption| Editing| **1)** Proportion of people blocked after publishing an edit where Reference Check was shown //and// **2)** Proportion of published edits that add new content and are reverted within 48hours|