User story & summary:
As a Product Manager or Product Data Analyst, I want to establish clear success and failure metrics so that we can determine whether to continue investing in this project or explore alternative solutions to the underlying problem.
Background
This task focuses on defining stopping rules—criteria for deciding when to pivot if the experiment does not achieve the expected impact after this iteration.
- Growth team project page: Constructive activation experimentation
Prior Research:
The "Add a link" experiment shows this Structured task leads to increases in newcomer participation, particularly by making constructive (non-reverted) article edits:
- an increase in the probability that newcomers make their first article edit (+16.6% over baseline, +10.1% over the unstructured add links task)
- And increase in the probability that they are retained as newcomers (+16.2% over baseline, +5.7% over the unstructured link task)
Decision:
Currently, "Add a Link" suggestions are unavailable for high-traffic pages, limiting exposure and the ability to gather sufficient data for statistically significant results. To address this, we are taking the following steps:
- Increase the “Add a Link” task pool: T386250: Rewrite refreshLinkRecommendations to not iterate through article topics
- Expand the target audience T388622: Increase target audience for Surfacing Structured Task Experiment
- Make “Add a Link” suggestions more noticeable: T388357: Second design iteration: Surfacing "Add a Link" in read view
Once these tasks are complete, we need clear goals and stopping rules to determine whether to continue investing in this approach or explore alternatives.
Acceptance Criteria:
- Product Manager and Product Data Analyst to draft stopping rules
- Product Manager to share and discuss with WE1.2 stakeholders