Page MenuHomePhabricator

Define selection criteria and measurement plan for GrowthExperiments + Central Notice Banner on small wikis
Closed, ResolvedPublic

Description

As part of T391035, one of the interventions is to enable growth experiments on a set of small wikis (likely around 5) and invite native speakers through a central notice banner campaign to contribute using these features.

Hypothesis statement:

If we invite native speakers of small wikis through a CentralNotice banner on a high-traffic Wikipedia in their region to contribute to SuggestedEdits and other Growth features, we can assess whether this approach attracts new native speakers and if they use these editing tools to improve vital content.

The idea is to try and enable as many growth features as feasible for a small wiki. Some of these GrowthFeatures are:

  • Help module, which will allow easy access to existing resources or resources to be developed.
  • Impact module, through which newcomers will be able to see the impact of their contributions more directly.
  • Community Updates module, which makes community events and initiatives more visible and accessible to new editors.
  • Possibly the Suggested and Mentorship modules, if maintenance templates and an admin are available on the wiki.

For selection criteria, it might be useful to look at:

  • Small wikis with a reasonable size of editors likely 5-25, where impact can be studied.
  • For the Structured Tasks and Mentorship modules to work, wikis need to have maintenance templates, and an admin must be available. So, we will focus on wikis that meet these two conditions.

For measurement, some things we could look at:

  • Number of accounts created
  • Number of people who used growth features
  • Number of editors retained from the experiment X days later
    • Note: Although, this is not necessary. It might be an important metric to measure as it will inform the potential user base for Q2 experiments
  • Number of people who clicked and viewed the banner. See T398646.

Measurement Plan: https://docs.google.com/document/d/1rBh_CTPe4JtxNUnE5dMImfzv-TKD3m4tbgz8CKzeq_M/edit?tab=t.0
Selected Wikis: https://docs.google.com/spreadsheets/d/1YeGDSOsaW8hBv_AiGfuQl_rCulB8PDZHQmocqV8HoYA/edit?gid=1711390606#gid=1711390606

Event Timeline

KCVelaga subscribed.
This comment was removed by KCVelaga.

Note: both the selection criteria (which wikis to select) and what we should measure as a primary metric are essentially blocked until the work from T390967 is complete (or near completion).

Even to select wikis, we will need the definitions of what is reasonable content growth and definition of sustainability, and the selection will be wikis which don't meet the defined criteria (+ any other considerations like templates). For evaluation, while the listed metrics are helpful for additional understanding (as secondary metrics), the primary metric will be dependent on the content growth & wiki sustainability definitions, as that is what we want to impact and change with these interventions.

MaryMunyoki added subscribers: KCVelaga_WMF, MaryMunyoki.

to be re-assigned once clear who will be working on this task

KCVelaga_WMF lowered the priority of this task from High to Medium.May 19 2025, 9:05 PM
srishakatux renamed this task from Define selection criteria and measurement plan for GrowthExperiments on small wikis to Define selection criteria and measurement plan for GrowthExperiments + Central Notice Banner on small wikis.Jul 8 2025, 5:34 AM
srishakatux assigned this task to cchen.
srishakatux updated the task description. (Show Details)
srishakatux added subscribers: ldelench_wmf, Aklapper.
cchen updated the task description. (Show Details)