Hypotheses and metrics
Edit Cards and the changes to the subsequent dialogs will:
hypothesis 1: ...make contributors more likely to start adding and modifying links and citations, since the actions to edit them will be more prominent on the page
metric 1: average number of link and citation workflow starts per session
hypothesis 2: ...make contributors more likely to finish adding and modifying links and citations, since the steps to do so will be more visible and clear.
metric 2: link and citation workflow completion rate
hypothesis 3: ...make contributors more likely to publish their edits, since it will be more clear when they have finished their task.
metric 3: edit completion rate
Instrumentation
Based on our post-deployment checks (T221254), we have detailed data on the linking and citation workflows starting 13 July. For details about the linking and citation workflows, see the feature instrumentation spec and the data dictionary.
Experiment plan
We will do a basic pre/post analysis of whether the metrics above changed due to the multiple iterative deployments of Edit Cards:
| version | date deployed |
|---|---|
| v1 | 21-June to Bengali, Hebrew, and Persian Wikipedias (T221314#5272510); v1 was never deployed to all wikis |
| v2 | 1-August 2019 in 1.34.0-wmf.16 (T225834) |
| v3 | 29-August 2019 in 1.34.0-wmf.20 (T229830) |
| v4 | 12-September-2019 in 1.34.0-wmf.22 (T231342) |
We should exclude any sessions that have a bucket set in EditAttemptStep, because those are part of the mobile-VE-as-default A/B test.
Glossary
- Edit cards: the proposed new design of the Context items on mobile. These "edit cards" will show details about, and actions related to, different elements within articles, like links, citations, images, infoboxes, templates, etc. (Developer Glossary)
- Context Item: a pop-up that appears anytime the cursor or focus is within the context of a specific node or annotation. It shows information, but doesn’t let you edit directly. Multiple context items can appear at once as a stack when one cursor position is in multiple contexts. (Developer Glossary)
- This is the pre-Edit Card implementation of mobile context items (i.e. the shelf that appears beneath the toolbar in mobile VE)
- Abort: a contributor has “discarded” their changes/exited the dialog
- Finish: a contributor has “applied” their changes after making changes to a dialog.
- Example: a contributor has successfully changed the label of an existing link.
- Publish: a contributor has made changes and successfully publishes those changes (i.e. they complete their edit(s)).
- Example: Adds new wiki link > writes edit summary > publishes edit.
- Changes: whether or not a contributor has typed in the dialog and to a rough extent, how much they’ve been typing
