Maniphest T419191

Start new A/A test for reader retention for March 2026
Closed, ResolvedPublic2 Estimated Story PointsBUG REPORT
Actions

Assigned To

Authored By

	KReid-WMF
	Mar 6 2026, 1:23 AM

Project Tags

Referenced Files

None

Subscribers

Aklapper

JMoore-WMF

KReid-WMF

Description

We would like to measure retention for logged out readers for March 2026 via an A/A test similar to T409190: Start new A/A test for retenion which will run for 21 days. To gather 21 days of data in March, the test needs to start early next week.

Requirements

Add collection of retention to WikimediaEvents as in T409190
Backport the collection code in order to enable collection to launch next week
Configure A/A test in TK UI with same parameters as previous measurements, to launch next week (March 8 or 9) and run for 21 days: https://mpic.wikimedia.org/experiment/logged-out-retention-round2

Details

Related Changes in Gerrit:

Subject	Author	Repo	Branch	Lines +/-
ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention	Phuedx	mediawiki/extensions/WikimediaEvents	wmf/1.46.0-wmf.18	+53 -43
JS SDK: Add getExperimentByPrefix()	Phuedx	mediawiki/extensions/TestKitchen	wmf/1.46.0-wmf.18	+147 -1
JS SDK: Add getExperimentByPrefix()	Phuedx	mediawiki/extensions/TestKitchen	master	+147 -1
ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention	Phuedx	mediawiki/extensions/WikimediaEvents	master	+53 -43

Customize query in gerrit

Related Objects

Mentioned In: T419914: Add ability to run multiple concurrent reader retention metrics tests
Mentioned Here: T409190: Start new A/A test for retenion

Event Timeline

KReid-WMF created this task.Mar 6 2026, 1:23 AM

KReid-WMF triaged this task as High priority.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 6 2026, 1:23 AM

KReid-WMF renamed this task from Start new A/A test for reader retenion for March 2026 to Start new A/A test for reader retention for March 2026.Mar 6 2026, 1:25 AM

phuedx claimed this task.Mar 6 2026, 8:37 AM

phuedx moved this task from Ready for Development to In Progress on the Test Kitchen (Experiment Platform Sprint 20) board.

phuedx set the point value for this task to 2.

Change #1248793 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/TestKitchen@master] JS SDK: Add getExperimentByPrefix()

https://gerrit.wikimedia.org/r/1248793

gerritbot added a project: Patch-For-Review.Mar 6 2026, 11:57 AM

Change #1248794 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/WikimediaEvents@master] ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention

https://gerrit.wikimedia.org/r/1248794

phuedx moved this task from In Progress to Needs Review on the Test Kitchen (Experiment Platform Sprint 20) board.Mar 6 2026, 12:28 PM

Change #1248793 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@master] JS SDK: Add getExperimentByPrefix()

https://gerrit.wikimedia.org/r/1248793

ReleaseTaggerBot added a project: MW-1.46-notes (1.46.0-wmf.19; 2026-03-10).Mar 9 2026, 10:00 AM

Change #1248794 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention

https://gerrit.wikimedia.org/r/1248794

Maintenance_bot removed a project: Patch-For-Review.Mar 9 2026, 10:30 AM

phuedx updated the task description. (Show Details)Mar 9 2026, 10:43 AM

phuedx updated the task description. (Show Details)Mar 9 2026, 10:59 AM

Change #1249242 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.18] ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention

https://gerrit.wikimedia.org/r/1249242

Change #1249243 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/TestKitchen@wmf/1.46.0-wmf.18] JS SDK: Add getExperimentByPrefix()

https://gerrit.wikimedia.org/r/1249243

Change #1249243 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@wmf/1.46.0-wmf.18] JS SDK: Add getExperimentByPrefix()

https://gerrit.wikimedia.org/r/1249243

Change #1249242 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.46.0-wmf.18] ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention

https://gerrit.wikimedia.org/r/1249242

Mentioned in SAL (#wikimedia-operations) [2026-03-09T14:05:55Z] <phuedx@deploy2002> Started scap sync-world: Backport for [[gerrit:1249243|JS SDK: Add getExperimentByPrefix() (T419191)]], [[gerrit:1249242|ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention (T419191)]]

Mentioned in SAL (#wikimedia-operations) [2026-03-09T14:07:46Z] <phuedx@deploy2002> phuedx: Backport for [[gerrit:1249243|JS SDK: Add getExperimentByPrefix() (T419191)]], [[gerrit:1249242|ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention (T419191)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-03-09T14:15:34Z] <phuedx@deploy2002> Finished scap sync-world: Backport for [[gerrit:1249243|JS SDK: Add getExperimentByPrefix() (T419191)]], [[gerrit:1249242|ext.wikimediaEvents: pageVisit -> loggedOutReaderRetention (T419191)]] (duration: 09m 39s)