Page MenuHomePhabricator

ImageBrowsing: instrumentation data QA
Closed, ResolvedPublic3 Estimated Story Points

Description

Instrumentation has been implemented in T403255: Instrument image browser's primary metric, T403259: Instrument image browsing interactions, and T405489: Instrument image browser's guardrail metric.

Acceptance criteria

[] Beta: verify that all instrumented events fire - blocked by T408095: The `X-Experiment-Enrollments` HTTP header isn't honored on the Beta cluster

    • page-visit for measuring retention isn't currently fired. It's implemented in WikimediaEvents, where imageBrowsingPageVisit.js isn't loaded. Submit a fix
  • Production
    • override enrollment as follows:
      • go to a target Wikipedia page - French example, Chinese example
      • in the browser's JavaScript console, override enrollment with mw.xLab.overrideExperimentGroup( 'fy2025-26-we3.1-image-browsing-ab-test', 'image-browsing-test' );. this should reload the page and enable image browsing

[] check they are all fully decorated with the experiment and experiments fields. - this isn't possible, since events are logged to the browser's console and only contain action and interactionData fields, as per docs

  • this isn't happening. Convert mw.eventLog.submitInteraction calls to Experiment#send()

[] check that all events are produced on the eqiad.mediawiki.product_metrics.readerexperiments_imagebrowsing and codfw.mediawiki.product_metrics.readerexperiments_imagebrowsing Kafka topics - this isn't possible.
[] check that all events are collected in the Data Lake at event.mediawiki_product_metrics_readerexperiments_imagebrowsing - this isn't possible.

NOTE: enrollment override won't write events, it will only log them in the browser's console. More details in this thread and initial discussion. CC @jwang.

Event Timeline

mfossati renamed this task from Image Browsing: Smoke Test / Data QA to ImageBrowsing: instrumentation data QA.Oct 16 2025, 3:33 PM
mfossati changed the task status from Open to In Progress.
mfossati triaged this task as High priority.
mfossati updated the task description. (Show Details)
mfossati added a subscriber: jwang.
egardner set the point value for this task to 3.Oct 16 2025, 4:14 PM

Change #1196730 had a related patch set uploaded (by Marco Fossati; author: Marco Fossati):

[mediawiki/extensions/WikimediaEvents@master] xLab: Load & execute Reader Growth's ImageBrowsing `page-visited` event

https://gerrit.wikimedia.org/r/1196730

Change #1196730 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] xLab: Load & execute Reader Growth's ImageBrowsing `page-visited` event

https://gerrit.wikimedia.org/r/1196730

Change #1196902 had a related patch set uploaded (by Marco Fossati; author: Marco Fossati):

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: send instrumented events via xLab

https://gerrit.wikimedia.org/r/1196902

Change #1196908 had a related patch set uploaded (by Marco Fossati; author: Marco Fossati):

[mediawiki/extensions/MetricsPlatform@master] xLab: include ReaderExperiments' ImageBrowsing stream

https://gerrit.wikimedia.org/r/1196908

Change #1196908 merged by jenkins-bot:

[mediawiki/extensions/MetricsPlatform@master] xLab: include ReaderExperiments' ImageBrowsing stream

https://gerrit.wikimedia.org/r/1196908

Change #1196902 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: send instrumented events via xLab

https://gerrit.wikimedia.org/r/1196902

While I'll take care of verification on Beta, I'm handing this over to @jwang for final QA & signoff.

mfossati updated Other Assignee, removed: jwang.

@mfossati - could you give an update on what QA for this ticket was?

@ovasileva this ticket has encompassed several discussions with Experiment Platform and ultimately covered data QA on:

  • a local dev environment (2nd iteration, i.e., mock enrollment via the HTTP header method)
  • the Beta cluster via enrollment override
  • production via enrollment override

Since enrollment override doesn't actually fire events, I've filed a follow-up ticket for the final end-to-end QA: T408104: ImageBrowsing: instrumentation production data QA. This needs to be done after T408105: ImageBrowsing: launch the A/B test on first set of wikis.
Closing!