Page MenuHomePhabricator
Create Task
Maniphest T409163

Set up measurement plan and instrumentation spec for mobile sticky headers
Closed, ResolvedPublic
Actions

Description

Background

We're planning on running an experiment to compare the new mobile sticky header design to the existing default behavior on the mobile site. (

Acceptance criteria

  • Set up measurement plan and instrumentation spec for mobile sticky headers
Metrics:

Primary:

  • Session length

Secondary:

  • Number of clicks on section headers
  • CTR for sections

Guardrail:

  • Retention

Design

Prototype: https://vector-2022.vercel.app/sticky-section-headers (note that this prototype shows sections as collapsed by default. We would expect sections to be open by default. To view this behavior, open all sections manually, then scroll to the top or try https://af619beaf3.catalyst.wmcloud.org/wiki/Sticky_Headers_Test?stickyHeaders=1)

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Instrument sticky header page visits & togglesmediawiki/extensions/WikimediaEventsmaster+146 -43
[WIP] Instrument Sticky Headersmediawiki/extensions/WikimediaEventsmaster+56 -2
Customize query in gerrit

Related Objects

Event Timeline

ovasileva created this task.Nov 4 2025, 11:54 AM
ovasileva triaged this task as High priority.
HSwan-WMF moved this task from Incoming/Inbox to Ready on the Reader Growth Team board.Nov 5 2025, 5:20 PM
jwang edited projects, added Product-Analytics (Kanban); removed Product-Analytics.Nov 10 2025, 5:33 PM
jwang mentioned this in T408047: MobileSections: Determine our instrumentation plan.Nov 10 2025, 6:03 PM
SherryYang-WMF moved this task from Ready to Sprint 4 (Nov 12 - Nov 25) Q2 25/26) on the Reader Growth Team board.Nov 12 2025, 5:33 PM
SherryYang-WMF edited projects, added Reader Growth Team (Sprint 4 (Nov 12 - Nov 25) Q2 25/26)); removed Reader Growth Team.
• mfossati mentioned this in T409956: Spike on instrumenting session length.Nov 13 2025, 12:02 PM
jwang moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.Nov 18 2025, 5:17 PM
SherryYang-WMF added a comment.Nov 20 2025, 10:38 PM

for the question in the measurement plan GoogleDoc, devs aligned on proposal one, bucketing page length in bytes by orders of magnitude. discussion on Slack: https://wikimedia.slack.com/archives/C08TB64J750/p1763492173335759

jwang added a comment.Nov 22 2025, 1:39 AM

Status update
Here are the drafts of the measurement plan and the instrumentation spec. For the user flow, please refer to Mobile section header Instrumentation and workflow.

HSwan-WMF moved this task from Sprint 4 (Nov 12 - Nov 25) Q2 25/26) to Sprint 5 (Nov 25 - Dec 9) Q2 25/26 on the Reader Growth Team board.Nov 25 2025, 5:10 PM
HSwan-WMF edited projects, added Reader Growth Team (Sprint 5 (Nov 25 - Dec 9) Q2 25/26); removed Reader Growth Team (Sprint 4 (Nov 12 - Nov 25) Q2 25/26)).
gerritbot added a comment.Dec 1 2025, 4:25 PM

Change #1213525 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/WikimediaEvents@master] [WIP] Instrument Sticky Headers

https://gerrit.wikimedia.org/r/1213525

gerritbot added a project: Patch-For-Review.Dec 1 2025, 4:25 PM
gerritbot added a comment.Dec 3 2025, 8:22 PM

Change #1214630 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/WikimediaEvents@master] Instrument sticky header page visits & toggles

https://gerrit.wikimedia.org/r/1214630

gerritbot added a comment.Dec 3 2025, 8:22 PM

Change #1213525 abandoned by Matthias Mullie:

[mediawiki/extensions/WikimediaEvents@master] [WIP] Instrument Sticky Headers

Reason:

Superseded by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/1214630

https://gerrit.wikimedia.org/r/1213525

gerritbot added a comment.Dec 4 2025, 3:32 PM

Change #1214630 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/WikimediaEvents@master] Instrument sticky header page visits & toggles

https://gerrit.wikimedia.org/r/1214630

gerritbot added a comment.Dec 8 2025, 3:07 AM

Change #1214630 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Instrument sticky header page visits & toggles

https://gerrit.wikimedia.org/r/1214630

ReleaseTaggerBot added a project: MW-1.46-notes (1.46.0-wmf.7; 2025-12-16).Dec 8 2025, 4:00 AM
SherryYang-WMF moved this task from Sprint 5 (Nov 25 - Dec 9) Q2 25/26 to Sprint 6 (Dec 10 - EOY) Q2 25/26 on the Reader Growth Team board.Dec 10 2025, 4:22 PM
SherryYang-WMF edited projects, added Reader Growth Team (Sprint 6 (Dec 10 - EOY) Q2 25/26); removed Reader Growth Team (Sprint 5 (Nov 25 - Dec 9) Q2 25/26).
Volker_E subscribed.Dec 10 2025, 8:37 PM

In case this experiment would validate not collapsing sections by default, the outcome would also put T147376: Evaluate showing open section based on search engine referrer on mobile web to end.

SherryYang-WMF closed this task as Resolved.Dec 16 2025, 7:20 PM
matthiasmullie added a project: ReaderExperiments-StickyHeaders.Jan 6 2026, 1:30 PM
Restricted Application edited projects, added Reader Growth Team; removed Reader Growth Team (Sprint 6 (Dec 10 - EOY) Q2 25/26). · View Herald TranscriptJan 6 2026, 1:30 PM
Aklapper edited projects, added Reader Growth Team (Sprint 6 (Dec 10 - EOY) Q2 25/26); removed Reader Growth Team.Jan 6 2026, 2:04 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits