Prior to deploying the Replying tool to //all// volunteers at our four partner wikis (T249394), we would like to know how the tool is being used/adopted as a Beta Feature.
Reason being: we would like to decide whether the jump in usage we can anticipate between the feature being available as an opt-in Beta Feature and it being available as an opt-out user Preference is large enough to warrant us taking an intermediate deployment step. [2]
To help evaluate the "usage/adoption" we would like to understand the following "Adoption metrics."
=== Adoption metrics
//For each of our four partner wikis – Arabic, Dutch, French and Hungarian – we would like to know the following...//
**How many people should we expect to try the Replying feature when it is turned on as an opt-out user preference four all users, at our four partner wikis?**
- Upper bound: number of people who have made at least 1 edit, in any namespace, in the previous 30 day period=== Timing
- Lower bound: number of people who have made at least 1 edit in a talk namespace in the previous 30 day periodQ4/2019-2020
**From 31-March-2020 onward, how many people have had access to the Reply tool ?**
//Graphed over time, segmented by wiki//
- How many people have explicitly [3] turned on the DiscussionTools Beta Feature?
- How many people had the DiscussionTools Beta Feature turned on for them? [4]=== Adoption metrics
- How many people have turned off the DiscussionTools Beta Feature?//For each of our four partner wikis – Arabic, Dutch, French and Hungarian – we would like to understand the metrics described below. These metrics are sorted in order of priority: highest priority = 1; [5]lowest priority = 4.
**1. From 31-March-2020 onward, how many people have used the Reply tool?**
- How many people have made 1 edit w/ DiscussionTools?
- How many people have made 2-5 edits w/ DiscussionTools?
- How many people have made 5-10 edits w/ DiscussionTools?
- How many people have made 10+ edits w/ DiscussionTools?
- //[Nice to have] Of the people who have made at least one edit with the Reply tool, how many of these people have made >10%, >25% and >50% of their total talk page edits using it after turning on the feature?//
**2. From 31-March-2020 onward, how has peoples' usage of the Reply tool changed over time?**
- Of the people who tried the Reply tool in Week 0, what percentage of these people made at least 1 edit with the tool at any point during Week 1, Week 2, Week 3, Week 4, Week 5, etc.
=== Timing**3. From 31-March-2020 onward, how many people have had access to the Reply tool ?**
Q4/2019-2020//Graphed over time, segmented by wiki//
- How many people have explicitly [3] turned on the DiscussionTools Beta Feature?
- How many people had the DiscussionTools Beta Feature turned on for them? [4]
- How many people have turned off the DiscussionTools Beta Feature? [5]
**4. How many people should we expect to try the Replying feature when it is turned on as an opt-out user preference four all users, at our four partner wikis?**
- Upper bound: number of people who have made at least 1 edit, in any namespace, in the previous 30 day period
- Lower bound: number of people who have made at least 1 edit in a talk namespace in the previous 30 day period
=== Open questions
- [ ] **Retention:** how – if at all – should the way we calculate retention account for the fact there may be some weeks where people do not make any talk page edits?
=== Done
- [ ] The "Adoption metrics" above have been calculated
- //Once the initial report is created, ideally, the Editing Team will be able to re-run these queries independent of #product-analytics.//
---
1. https://www.mediawiki.org/wiki/Talk_pages_project/replying#Step_3:_User_Preference_(opt-out)
2. **Intermediate deployment step:** e.g. deploying the feature as an opt-out user Preference to 50% of contributors on target wikis.
3. **"Explicitly"**: meaning they //did not// have the following preference checked: `Automatically enable all new beta features`
4. **"Turned on for them"**: meaning they //did// have the following preference checked: `Automatically enable all new beta features`
5. This should include everyone, regardless of how the feature became available to them (e.g. whether they turned it on explicitly or whether it was automatically enabled for them)