Page MenuHomePhabricator

Measurement planning for Logged In Reader Retention Baseline A/A Test (Q4 FY25-26)
Closed, ResolvedPublic2 Estimated Story Points

Description

Background

We do not have an existing measurement of user retention rate for logged-in Wikipedia readers (defined currently as someone who has 0 article edit count across all wikis), due to instrumentation only capturing specific "user touch" events, such as logging in or making a change in user preferences. This does not capture a logged-in user simply browsing and reading Wikipedia.

We will use a 1 month long sampled A/A test in TestKitchen to log the return frequency of a cohort of randomized users to get a better understanding of logged in user retention over that period. This baseline will be useful in helping us set key result goals for improving logged-in and active user retention on web next FY.

We are aiming to run the data collection from April 1 to May 1, 2026, across the top 50 trafficked wikis, to get a representative population of user retention.


Requirements

Acceptance criteria

  • Jennifer draft the spec
  • Review jennifer's spec, share any questions or concerns

BDD

  • For QA engineer to fill out.

Test Steps

  • For QA engineer to fill out.

Communication criteria

Add if this needs an announcement or discussion.

Rollback plan

Describe the rollback plan in production for this task if something goes wrong.

This task was created by Version 1.0.0 of the Reader Experience team task template using phabulous.

Event Timeline

HFan-WMF updated the task description. (Show Details)
bwang set the point value for this task to 2.
bwang moved this task from Needs refinement to Ready for sprint on the Reader Experience Team board.
HFan-WMF renamed this task from Measurement spec for Logged In Reader Retention Baseline A/A Test (Q4 FY25-26) to Review measurement spec for Logged In Reader Retention Baseline A/A Test (Q4 FY25-26).Mar 25 2026, 6:12 PM
HFan-WMF removed jwang as the assignee of this task.
HFan-WMF added a subscriber: jwang.
jwang renamed this task from Review measurement spec for Logged In Reader Retention Baseline A/A Test (Q4 FY25-26) to Measurement planning for Logged In Reader Retention Baseline A/A Test (Q4 FY25-26).Mar 30 2026, 5:28 PM
jwang updated the task description. (Show Details)
HFan-WMF claimed this task.