Maniphest T399484

Verify that A/A test for retention baseline is working correctly via a "smoke test"
Closed, ResolvedPublic2 Estimated Story Points
Actions

Assigned To

Authored By

	egardner
	Jul 14 2025, 5:21 PM

Tags

Referenced Files

None

Subscribers

Description

Once T399347 and T399482 are complete, we should run a "smoke test" of our experiment on somewhere like TestWiki. This will allow us to confirm that we're segmenting traffic, collecting the necessary data, etc. The duration of this initial test does not need to be long and this can probably happen within one sprint.

Open questions

Do we need to do anything to manually segment users for this?

Acceptance Criteria

A config patch has been merged to enable the User Retention instrument on Test Wiki
The test has been turned off and we can confirm that things worked as expected

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		None	T399227 [Epic] Perform an A/A test for retention baseline
		Resolved		egardner	T399484 Verify that A/A test for retention baseline is working correctly via a "smoke test"

Event Timeline

egardner created this task.Jul 14 2025, 5:21 PM

egardner mentioned this in T399227: [Epic] Perform an A/A test for retention baseline.

egardner mentioned this in T399486: Turn on the A/A test in production.Jul 14 2025, 5:27 PM

ovasileva triaged this task as High priority.Jul 22 2025, 4:41 PM

ovasileva set the point value for this task to 2.Jul 22 2025, 4:50 PM

ovasileva moved this task from Incoming/Inbox to Ready on the Reader Growth Team board.

HSwan-WMF moved this task from Ready to Sprint 2 on the Reader Growth Team board.Jul 23 2025, 4:41 PM

HSwan-WMF edited projects, added Reader Growth Team (Sprint 2); removed Reader Growth Team.

KSarabia-WMF claimed this task.Jul 23 2025, 5:54 PM

KSarabia-WMF moved this task from Committed to Doing on the Reader Growth Team (Sprint 2) board.Jul 23 2025, 6:30 PM

jwang mentioned this in T400417: Data QA: retention instrumentation.Jul 24 2025, 10:08 PM

KSarabia-WMF moved this task from Doing to Signoff on the Reader Growth Team (Sprint 2) board.Jul 29 2025, 7:16 PM

egardner mentioned this in T398496: Image Browsing: Plan a "smoke test" to validate that analytics is working properly.Jul 30 2025, 5:45 PM

ovasileva reassigned this task from KSarabia-WMF to egardner.Aug 5 2025, 4:10 PM

egardner closed this task as Resolved.Aug 5 2025, 5:21 PM

Jdlrobson-WMF added a project: FY2025-26 WE3.6 Consumer Strategy.Oct 14 2025, 11:32 PM