Page MenuHomePhabricator

[EPIC] Surface Structured Tasks to new editors (FY24/25: WE1.2.3 & WE1.2.6 & WE1.2.9)
Closed, ResolvedPublic

Description

User story & summary:

As a new editor, I need guidance and structure when completing my first edit on wiki, because then I'm more likely to constructively activate (complete my first edit that isn't reverted).

WMF Annual Plan:

This Epic relates to WE1.2 from FY24-25:
Current full-page editing experiences require too much context, patience, and trial and error for many newcomers to contribute constructively. To support a new generation of volunteers, we will increase the number and availability of smaller, structured, and more task-specific editing workflows (E.g. Edit Check and Structured Tasks).

Related: T360829: WE 1.2: Establish baseline for constructive activation

Background & research:

The Add an Image experiment shows this Structured task leads to an increase in newcomer participation on the mobile web platform, particularly by making constructive (non-reverted) article edits:

  • The likelihood that mobile web newcomers make their first article edit (+17.0% over baseline)
  • The likelihood that they are retained as newcomers (+24.3% over baseline)

The "Add a link" experiment shows this Structured task leads to increases in newcomer participation, particularly by making constructive (non-reverted) article edits:

  • an increase in the probability that newcomers make their first article edit (+16.6% over baseline, +10.1% over the unstructured add links task)
  • And increase in the probability that they are retained as newcomers (+16.2% over baseline, +5.7% over the unstructured link task)
Hypotheses:

Currently new editors can only find Structured tasks via their Homepage, but we hypothesize that Structured tasks could be even more meaningful to newcomers (and have an even greater impact on newcomer activation and retention) if we surface these tasks to new editors in the moment they are needed. For example, if a suggestion is surfaced to the new editor when they navigate to edit an article that interests them.

Strategize & Discover (Q1, FY24-25) WE1.2.3:
✅ If we design and user test two or more design prototypes in which we surface a structured task to newcomers in Visual Editor, then we can quickly learn and iterate on designs before investing in engineering.
✅ If Growth engineers complete a technical research spike into the feasibility of the design prototype UX, then we can provide estimates into the effort of the different approaches.

Define & Develop (Q2, FY24-25) WE1.2.6:
✅ If we release a way for people to act on “Add a link” suggestions from an article in Wikipedia to Beta,
✅ then we will set the stage for an experiment in early Q3, where we aim to increase the percentage of new account holders who constructively activate on mobile by 10% compared to the baseline.

Deliver & Measure (Q3, FY24-25) WE1.2.9:
✅ (Mar 4, 2025) If we surface the ‘Add a Link’ Structured Task to new account holders who are reading Wikipedia articles through an A/B test on pilot wikis T385343
⌛ Then we expect to increase the percentage of these people who constructively activate on mobile. T377098

Release Plan:

Q2 Alpha Release:
T379976: Surfacing "Add a link" Structured Tasks: Alpha Release Plan and Release Task (FY24/25 WE1.2.6)
We hope to work in an agile way to release a simple MVP ASAP to pilot wikis. This will be a temporary release where we release for a temporary time to learn fast and then roll back before the December release freeze / holidays. We will limit this release to only new account holders with zero edits on mobile.

Q3 Pilot Experiment Release:
T385343: Surfacing "Add a link" Structured Tasks: Experiment Release (FY24/25 WE1.2.9)
We will iterate based on what we learn from the Q2 Alpha Release and start a an official experiment in early Q3 on two or more pilot wikis.

Scaling to more Wikipedias:
TBD; rolling back or scaling to more wikis will depend on results from the Pilot Experiment Release.

Scope of this Epic:

This epic includes the initial work to surface structured tasks to new account holders, and the associated experiment analysis.

Related Objects

StatusSubtypeAssignedTask
ResolvedKStoller-WMF
OpenNone
ResolvedKStoller-WMF
ResolvedJFernandez-WMF
ResolvedJFernandez-WMF
ResolvedJFernandez-WMF
ResolvedJFernandez-WMF
ResolvedJFernandez-WMF
ResolvedJFernandez-WMF
ResolvedCyndymediawiksim
ResolvedCyndymediawiksim
ResolvedMichael
ResolvedMichael
ResolvedJFernandez-WMF
DeclinedNone
ResolvedMichael
ResolvedMichael
Resolved Iflorez
ResolvedSgs
Resolved Iflorez
Resolved Iflorez
ResolvedKStoller-WMF
ResolvedKStoller-WMF
ResolvedUrbanecm_WMF
ResolvedCyndymediawiksim
ResolvedMichael
ResolvedJFernandez-WMF
DuplicateJFernandez-WMF
DeclinedNone
ResolvedJFernandez-WMF
ResolvedSgs
ResolvedUrbanecm_WMF
ResolvedMichael
ResolvedBUG REPORTMichael
ResolvedUrbanecm_WMF
ResolvedMichael
ResolvedUrbanecm_WMF
ResolvedMarostegui
ResolvedSgs
ResolvedTrizek-WMF
ResolvedEtonkovidova
ResolvedSgs
ResolvedMichael
ResolvedMichael
ResolvedMichael
ResolvedSgs
ResolvedJFernandez-WMF
ResolvedJFernandez-WMF
ResolvedMichael
ResolvedEtonkovidova
ResolvedUrbanecm_WMF
DeclinedBUG REPORTNone
ResolvedSgs
In ProgressMichael
OpenNone
OpenNone
OpenNone
DeclinedKStoller-WMF
ResolvedUrbanecm_WMF

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
KStoller-WMF renamed this task from (EPIC) Surface Structured Tasks to new editors within or alongside VisualEditor to [EPIC] Surface Structured Tasks to new editors within or alongside VisualEditor.Jun 5 2024, 9:52 PM
KStoller-WMF raised the priority of this task from Medium to High.Jul 24 2024, 5:01 PM
KStoller-WMF renamed this task from [EPIC] Surface Structured Tasks to new editors within or alongside VisualEditor to [EPIC] Surface Structured Tasks to new editors within or alongside VisualEditor (WE1.2.3).Jul 24 2024, 5:02 PM
KStoller-WMF renamed this task from [EPIC] Surface Structured Tasks to new editors within or alongside VisualEditor (WE1.2.3) to [EPIC] Surface Structured Tasks to new editors within or alongside VisualEditor (FY24/25 WE1.2.3).Jul 24 2024, 5:04 PM
KStoller-WMF claimed this task.
KStoller-WMF renamed this task from [EPIC] Surface Structured Tasks to new editors within or alongside VisualEditor (FY24/25 WE1.2.3) to [EPIC] Surface Structured Tasks to new editors within or alongside VisualEditor (FY24/25 WE1.2.3 & WE1.2.6).Oct 7 2024, 10:55 PM
KStoller-WMF updated the task description. (Show Details)
KStoller-WMF renamed this task from [EPIC] Surface Structured Tasks to new editors within or alongside VisualEditor (FY24/25 WE1.2.3 & WE1.2.6) to [EPIC] Surface Structured Tasks to new editors (FY24/25: WE1.2.3 & WE1.2.6 & WE1.2.9).Jan 31 2025, 10:22 PM

Project update on MediaWiki: https://www.mediawiki.org/wiki/Growth/Constructive_activation_experimentation#Surfacing_Structured_Tasks_in_Read_Mode_Pilot_Experiment

Final experiment results shared here:
https://analytics.wikimedia.org/published/reports/growth/structured_task_add_a_link_read_mode_ab_test_report.html

Conclusion: Since the results were inconclusive but showed a negative trend we have stopped the experiment and will not roll the feature out more widely. Further learnings are shared here; we will use these learnings to guide future experiments.