This test is intended to help us understand whether the new Reply tool is having the impact we intended it to haveat impact the Reply Tool is having on Junior Contributors' likelihood to start (activation) and continue (retention) participating on Wikipedia talk pages.
Determining whether the Reply tool is having the impact we intended it to will help us decide:===Decision to be made
1. Should the Reply tool be deploy//The decision this analysis is intended to all wikis?help us make://
2.**Should the Reply tool be offered to all people, at all wikis, as an //opt-out// user preference? What – if any – parts of the Reply tool should we consider revising?
=== Metrics**
=== Hypotheses
//To help evaluate the impact of the Reply tool, we would like to analyze whether adding an obvious and intuitive for replying to specific comments to Wikipedia talk pages causes the following://
To help evaluate the impact of the Reply tool, we would like to analyze the following metrics:|ID|Hypothesis|Metric(s) for evaluation|
- //@ppelberg to populate//
We would like to be able to compare how the above metrics vary by namespaces|---|---|---
|1.| A greater number of Junior Contributors to participate productively on talk pages.| The number of distinct Junior Contributors who make at least one edit to a page in a talk namespace that is not reverted within 48 hours. |
|2.| A greater percentage of Junior Contributors //continue// to participate productively on talk pages.| The percentage of Junior Contributors who who make at least one edit to a page in a talk namespace that is not reverted within 48 hours in each of the following time intervals: **2 to 7 days after making their edit** (read: within the first week), **8 to 14 days after making their first edit** (read: within the second week), wikis,and **15 to 30 days after making their first edit** (read: within the third or fourth weeks)
|3.| An increase in the number of disruptive edits being made to talk pages| The number of edits made to talk pages that are reverted within 48 hours. platforms and editor experienceThe number of editors who are blocked after making an edit to a talk page.
//Note: while hypotheses "1." and "2." specify "Junior Contributors," we would like to be able to look at the metrics associated with each of these hypotheses by experience level. [i]//
=== Open questions
- [ ] 1. Should edits to non-talk namespace pages be included in this analysis?
- [ ] 2. What wikis should be included in the test? //See: T267379.//
---
i. Editor experience buckets
- Logged out
- 0 cumulative edits
- 1-4 cumulative edits
- 5-99 cumulative edits
- 100-999 cumulative edits
- 1000+ cumulative edits