Background
We want to run a monthlong cohort tracking through an A/A test in Test Kitchen (similar to one already performed for logged out users) to see the current retention rate of logged-in users. We will run this A/A tracking from April 1, 2026 to May 1, 2026
User story
As the analyst, I want to be have access to a month's worth of data tracking each time a user account loads a page on wikipedia while logged in. This will give me the ability to analyze the retention rate over various timeframes, as well as focus on data from users who are in our target demographic (readers).
Requirements
Technical requirements
- (See patch.) Create and deploy a new event stream that includes the proper contextual attributes. The web_base stream contains page IDs which we do not need.
- (See patch.) Create a new loggedInReaderRetention instrument in WikimediaEvents. This will be similar to the loggedOutReaderRetention instrument created last year by the ReaderGrowth team.
- Ensure that visits by temp accounts are not logged
- (See test.) Create a new A/B test in the TestKitchen UI. See the instrumentation spec linked above for details. Yes, it's an A/A test, but it's configured as a new A/B test with a second control group.
- Test locally to confirm the instrument is working
- Turn on the instrument (at least 24 hours before the start date of 2026-04-01)
- Confirm that data is being gathered as expected
Acceptance criteria
- The A/A experiment instrument to track users per sample rate outlined in the spec is set up for the top 50 trafficked wikipedias
- We begin tracking this cohort on April 2, 2026 and run for 1 month
- Page-visited data per user ID is recorded (NOT page load history, only that any page is loaded)
- Temp accounts are not included
BDD
- For QA engineer to fill out.
Test Steps
- For QA engineer to fill out.
Communication criteria
Add if this needs an announcement or discussion.
Rollback plan
The experiment can be turned off in the TestKitchen UI if there are any issues.
This task was created by Version 1.0.0 of the Reader Experience team task template using phabulous.