In T404600, we are running a controlled experiment of Suggestion Mode to evaluate how it impacts new(er) volunteers (≤100 cumulative edits) .
This task holds the work of doing a complimentary analysis to understand how people who have enabled the Suggestion Mode beta feature are engaging with the feature.
Decision(s) to be made
- 1. Will Suggestion Mode (in its current from) be graduated from a beta feature to default-on and for whom?
- Note: this decision will also be impacted by the results of T404600.
Research questions
Note: all metrics ought to exclude volunteers bucketed into the controlled experiment and ideally, broken out by wiki and platform.
- Adoption: how many volunteers explicitly enabled the feature?
- Metric(s): count of unique users who enabled the Suggestion Mode Beta Feature.
- Conversion: how useful/reliable are the suggestions people are seeing?
- Metric(s): of the edit sessions in which volunteers saw ≥1 suggestion, what % said sessions resulted in a volunteer publishing an edit in which they accepted a suggestion?
- Retention / stickiness: are volunteers finding the feature enduringly useful?
- Metric: among experienced volunteers who published ≥1 edit in which they acted on a suggestion, what proportion return to publish another such edit in any of the following 2 weeks?