Page MenuHomePhabricator

FY 2025/26 WE 1.1: Set percentage improvement target for constructive edits
Closed, ResolvedPublic

Description

This task involves the work of setting the amount by which WE 1.1 will seek to increase constructive edits by the end of Q2, 2025 - 2026.

Requirements

Propose an amount by which WE 1.1 will seek to increase constructive edits by the end of Q2, 2025 - 2026, as measured through controlled experiments.

Where "constructive edits" are defined as = edits to pages in any Wikipedia main namespace that are not reverted within 48 hours of being published by people who have published ≤100 cumulative edits.

Methodology

To set an improvement target, we will extrapolate from the following pieces of information:

  1. The interventions we are likely to prioritize work on during Q1 and Q2 (let's call them "prioritized interventions")
  2. Of the interventions we're prioritizing:
    1. At what wikis we assume these prioritized interventions are likely to be made available to new(er) volunteers through controlled experiments
    2. By when said prioritized interventions will be made available
    3. The proportion of new(er) volunteers who will gain access to these prioritized interventions
      • @ppelberg: I'm proposing that we remove this consideration seeing as how we are committing to evaluating each individual intervention's impact on constructive edits, as measured through controlled experiments. This step/requirement would've been relevant had we been measuring the KR in terms of overall/global shifts in constructive edits.
    4. What past experiments the prioritized interventions are similar enough to that we can use said interventions' impact on constructive edits to make an assumption about how much these 2025-2026 interventions are likely to impact CA
NOTE: we are using the WE 1.1 Constructive edits KR research sheet to work through the above.

Steps

  1. Megan to prepare spreadsheet to hold the information described in the === Methodology section above
  2. Peter to populate the sheet with "prioritized interventions" and the expected deployment scope and timing
  3. Megan to extrapolate from "2." to propose an improvement target

Event Timeline

Aklapper renamed this task from WE 1.1: Set percentage improvement target for constructive edits (FY 25/26) to WE 1.1: Set percentage improvement target for constructive edits (FY 2025/26).Mar 19 2025, 6:38 PM

@ppelberg: Which team(s) is this task and its subtasks about? Please add project tags so this can be found on a workboard - thanks!

ppelberg updated the task description. (Show Details)
ppelberg added a project: Product-Analytics.

@ppelberg
Initial proposed target for mobile web: 4% relative increase

This is based on targets achieved by similar past interventions and anticipated natural year-over-year fluctuations. See summary of insights used to inform this target below.

Rationale:

  • In T389404, we established a baseline of 71% for mobile web based on the most recent three months of available data. To achieve a target of 4%, we would need to increase constructive edit rates to about 74%.
  • We should account for a natural increase in constructive edits of around 1-2% based on past trends in this metric[1].
    • Mobile web constructive edit rates have been fairly stable over the last three years, with slight YoY increases.
    • The average YoY 3-month rolling change is +1.46%
    • The most recent data (ending April 2025) shows a YoY 3-month rolling average increase of +1.81%. This indicates a continued positive trend in constructive edits.
  • Available data from past interventions with positive impacts on mobile web constructive rates have achieved around 3% increases for the targeted group (users with < 100 edits or unregistered users). This includes Reference Check, which shares a number of similar features to planned interventions.

[1] See monthly and yearly trend data summarized in T389404#10948162 and reviewed in this doc

Note: As discussed with @ppelberg, this baseline and target is based on trends across all Wikipedias and does not reflect per-wiki variances that may impact results for this metric if reviewed for a specific Wikipedia or subset of Wikipedias. There is an opportunity to revise this improvement target, if/when we come to learn interventions will be deployed at specific wikis.

Aklapper renamed this task from WE 1.1: Set percentage improvement target for constructive edits (FY 2025/26) to FY 2025/26 WE 1.1: Set percentage improvement target for constructive edits.Jun 30 2025, 7:25 PM

We set a target of 4% relative increase in mobile web constructive edits based on the analysis summarized in T389403#10960480. This was documented on the Annual Plan page.

Resolving this task but please reopen if needed.