Page MenuHomePhabricator

Experiment instrumentation for Mobile Page Previews
Closed, ResolvedPublic5 Estimated Story Points

Description

Implement event logging for the Mobile Page Previews experiment (T418812) so we can measure the feature's impact and evaluate success criteria.

Instrumentation

Measurement plan: https://docs.google.com/document/d/1dEtxCa0uItu56PChbw5m1IsrkXhTsPwGdBgeZrweXio/edit?usp=sharing
Instrumentation spec: https://docs.google.com/spreadsheets/d/13J9g4zcd4qjq7wtBAuJ8cyPVfYp-qsZ3f_P5n-JGRjc/edit?usp=sharing

Context

This is an A/B test delivered via the Test Kitchen framework. We need instrumentation to track the user funnel and support the experiment analysis.

Metrics:

Primary: Retention
Secondary:

  • Number of distinct pages interacted with per session
  • Number of clicks on blue links per session (if it's significantly easier to instrument than the above)

Curiosity:

  • Pageviews
  • Session length

Funnel events

https://www.figma.com/design/HTqNO7VJ81qeJYzvHbcoJY/Minerva-Page-previews?node-id=46-530&p=f&t=CUsv3O7ZdrqlTE5j-0

We probably want to capture the following events as they happen:

  • Link tap (qualifying link tapped, preview flow initiated)
  • Preview shown (bottom sheet opened with content visible)
  • Article opened (reader tapped "Open article" to navigate)
  • Preview dismissed (reader closed the sheet without navigating — via X, outside tap, or back button)

Additional signals to consider:

  • Time spent viewing the preview before action (open or dismiss)
  • Whether the previewed article had an image or not
  • API latency (time from tap to content rendered)
  • API errors (request failed or timed out)

Other things we need to capture?
-link clicks in the control group (click throughs to the target page)
-Jennifer's green light
-do gud dezign

Details

Event Timeline

egardner moved this task from Incoming/Inbox to Backlog on the Reader Growth Team board.
egardner moved this task from Backlog to Needs Refinement on the Reader Growth Team board.
HSwan-WMF set the point value for this task to 5.
SherryYang-WMF raised the priority of this task from Medium to High.Mar 31 2026, 10:31 PM

Change #1269001 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/ReaderExperiments@master] [WIP] Instrument page previews

https://gerrit.wikimedia.org/r/1269001

Change #1269001 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] Instrument page previews

https://gerrit.wikimedia.org/r/1269001