This test is intended to help us understand what impact the Reply Tool is having on Junior Contributors' likelihood to start (activation) and continue (retention) participating on Wikipedia talk pages.
===Decision to be made
//The decision this analysis is intended to help us make://
**Should the Reply tool be offered to all people, at all wikis, as an //opt-out// user preference? **
=== Hypotheses
//To help evaluate the impact of the Reply tool, we would like to analyze whether adding an obvious and intuitive for replying to specific comments to Wikipedia talk pages causes the following://
|ID|Hypothesis|Metric(s) for evaluation|
|---|---|---
|1.| A greater number of Junior Contributors to participate productively on talk pages.| The number of distinct Junior Contributors who make at least one edit to a page in a talk namespace that is not reverted within 48 hours. |
|2.| A greater percentage of Junior Contributors //continue// to participate productively on talk pages.| The percentage of Junior Contributors who who make at least one edit to a page in a talk namespace that is not reverted within 48 hours in each of the following time intervals: **2 to 7 days after making their edit** (read: within the first week), **8 to 14 days after making their first edit** (read: within the second week), and **15 to 30 days after making their first edit** (read: within the third or fourth weeks)
|3.| An increase in the number of disruptive edits being made to talk pages| The number of edits made to talk pages that are reverted within 48 hours. The number of editors who are blocked after making an edit to a talk page.
//Note: while hypotheses "1." and "2." specify "Junior Contributors," we would like to be able to look at the metrics associated with each of these hypotheses by experience level. [i]//
=== Open questions
- [ ] 1. Should edits to non-talk namespace pages be included in this analysis?
- [ ] 2. What wikis should be included in the test? //See: T267379.//
---
i. Editor experience buckets
- Logged out
- 0 cumulative edits
- 1-4 cumulative edits
- 5-99 cumulative edits
- 100-999 cumulative edits
- 1000+ cumulative edits