Topic Containers (T269950), Clearer Talking Affordances (T255560 + T267444), and Page Frame (T269963) are all interventions designed to help:
- Junior Contributors to quickly recognize talk pages as places to communicate with other volunteers and locate the tools available to do so
- Senior Contributors to be able to quickly assess which conversations on a given talk page are worth focusing on
This task involves the work with running an A/B test to evaluate the extent to which this set of Usability Improvements have been effective at impacting Junior and Senior Contributors in the ways described above.
Decision to be Made
This A/B test will help us make the following decision: Are the set of Usability Improvements on desktop fit to be made available to everyone, at all Wikimedia wikis, by default?
Hypotheses
| ID | Hypothesis | Metric(s) for Evaluation |
|---|---|---|
| KPI | Volunteers across experience levels and account states (logged in + out) will use talk pages in ways that align with their consented upon purpose (read: collaborate to improve Wikipedia) | 1) Proportion of published talk page edits, of all types, that are reverted, 2) Proportion of talk page edits that are started and successfully published and 3) Proportion of people who publish talk page edits and are subsequently blocked |
| Curiosity #1 | Junior Contributors will intuitively understand talk pages, across namespaces, as tools they can use to communicate with other volunteers because they will recognize other people using these pages to talk and identify the affordances they can use to do the same. | 1) Of all Junior Contributors that post on a talk page, the average time duration from when a Junior Contributor views a talk page to when they engage on the page in some way (for example, click an affordance to comment or start a conversation), 2) Of all Junior Contributors that visit a talk page, the proportion who engage with the page in some way |
| Curiosity #2 | A greater percentage of Junior Contributors who visit talk pages will publish at least one non-reverted comment or new discussion because they will be more clear and confident about what talk pages are used for and how to use them. | 1) The proportion of Junior Contributors who click an affordance on a talk page (defined as an init event being emitted) and successfully publish at least one comment or new discussion (defined as saveSuccess event being emitted) that is not reverted within 48 hours, 2) The proportion of Junior Contributors that visit a talk page and successfully publish at least one comment or new discussion (not reverted) |
| Curiosity #3 | Senior Contributors will be able to more quickly and easily decide where to focus their attention when arriving on a talk page because they will be able to see, at a glance, what new comments and/or discussions have been added since they last visited. | Of all Senior Contributors that post on a talk page, the average time from when a Senior Contributor visit a talk page to starting any new edit on a talk page. |
Guardrails
| ID | Name | Metric(s) for Evaluation |
|---|---|---|
| Guardrail #1 | Regressions in bounce rate | Bounce rate |
Decision Matrix
| ID | Scenario | Plan of Action |
|---|---|---|
Done
- Finalize ===Hypotheses and ===Decision matrix
- Complete analysis
- Publish findings on mediawiki.org: Talk pages project/Usability
