Page MenuHomePhabricator

Instrumentation for Synthetic A/A Test (SDS 2.4.11)
Closed, ResolvedPublic2 Estimated Story Points

Description

As per the measurement plan and instrumentation spec, the Synthetic A/A test will track page visits.

Acceptance Criteria:

  • Create an instrument named PageVisit that tracks the action page-visited
  • Delete the product_metrics.web_base.experiment_enrollment_handling.dev0 stream and request deletion of the associated table
  • Delete the product_metrics.web_base.experiment_enrollment_handling.dev1 stream and request deletion of the associated table
  • For beta cluster support, add the product_metrics.web_base stream
  • For beta cluster support, enable experimentation by setting MetricsPlatformEnableExperiments to true
  • For beta cluster support, send experiment-related analytics events to /evt-103e/v2/events by setting MetricsPlatformExperimentEventIntakeServiceUrl
  • For beta cluster support, enable testers to provide manual experiment overrides by setting MetricsPlatformEnableExperimentOverrides to true
  • TBD: For beta cluster support, add an experiment configuration corresponding to the PageVisit experiment to mediawiki-config (wgMetricsPlatformInstrumentConfiguratorBaseUrl would not presently be meaningful for beta cluster given domain name incongruence)

Event Timeline

mpopov renamed this task from Create Instrumentation for Synthetic A/A Experiment to Instrumentation for Synthetic A/A Test (SDS 2.4.11).May 12 2025, 3:37 PM
mpopov triaged this task as Medium priority.
mpopov edited projects, added Product-Analytics; removed Epic.
mpopov updated the task description. (Show Details)
mpopov moved this task from Triage to Tracking on the Product-Analytics board.
phuedx raised the priority of this task from Medium to High.May 22 2025, 3:14 PM
phuedx set the point value for this task to 2.May 23 2025, 5:59 PM

Change #1149709 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/WikimediaEvents@master] ext.wikimediaEvents: Add XLab PageVisit instrument

https://gerrit.wikimedia.org/r/1149709

Change #1151148 had a related patch set uploaded (by Phuedx; author: Phuedx):

[operations/mediawiki-config@master] EventStreamConfig: Remove xLab development streams

https://gerrit.wikimedia.org/r/1151148

Change #1151150 had a related patch set uploaded (by Phuedx; author: Phuedx):

[integration/config@master] Zuul: [mediawiki/extensions/WikimediaEvents] Add MetricsPlatform dependency

https://gerrit.wikimedia.org/r/1151150

Change #1151150 merged by jenkins-bot:

[integration/config@master] Zuul: [mediawiki/extensions/WikimediaEvents] Add MetricsPlatform dependency

https://gerrit.wikimedia.org/r/1151150

We may need to change the experiment name, depending on what is allowed in xLab.

Change #1149709 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] ext.wikimediaEvents: Add XLab PageVisit instrument

https://gerrit.wikimedia.org/r/1149709

Change #1152115 had a related patch set uploaded (by Clare Ming; author: Phuedx):

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.3] ext.wikimediaEvents: Add XLab PageVisit instrument

https://gerrit.wikimedia.org/r/1152115

Change #1151148 merged by jenkins-bot:

[operations/mediawiki-config@master] EventStreamConfig: Remove xLab development streams

https://gerrit.wikimedia.org/r/1151148

Mentioned in SAL (#wikimedia-operations) [2025-05-29T21:10:25Z] <cjming@deploy1003> Started scap sync-world: Backport for [[gerrit:1151148|EventStreamConfig: Remove xLab development streams (T393918)]]

Mentioned in SAL (#wikimedia-operations) [2025-05-29T21:12:18Z] <cjming@deploy1003> cjming, phuedx: Backport for [[gerrit:1151148|EventStreamConfig: Remove xLab development streams (T393918)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-05-29T21:20:00Z] <cjming@deploy1003> Finished scap sync-world: Backport for [[gerrit:1151148|EventStreamConfig: Remove xLab development streams (T393918)]] (duration: 09m 34s)

Change #1152115 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.3] ext.wikimediaEvents: Add XLab PageVisit instrument

https://gerrit.wikimedia.org/r/1152115

Mentioned in SAL (#wikimedia-operations) [2025-05-29T21:28:32Z] <cjming@deploy1003> Started scap sync-world: Backport for [[gerrit:1152115|ext.wikimediaEvents: Add XLab PageVisit instrument (T393918 T392313)]]

Mentioned in SAL (#wikimedia-operations) [2025-05-29T21:30:25Z] <cjming@deploy1003> cjming: Backport for [[gerrit:1152115|ext.wikimediaEvents: Add XLab PageVisit instrument (T393918 T392313)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-05-29T21:38:26Z] <cjming@deploy1003> Finished scap sync-world: Backport for [[gerrit:1152115|ext.wikimediaEvents: Add XLab PageVisit instrument (T393918 T392313)]] (duration: 09m 54s)

Change #1152253 had a related patch set uploaded (by Dr0ptp4kt; author: Dr0ptp4kt):

[operations/mediawiki-config@master] WIP DNM: Support edge uniques A/B tests on beta cluster

https://gerrit.wikimedia.org/r/1152253

Change #1152253 merged by jenkins-bot:

[operations/mediawiki-config@master] Beta Cluster: Support A/B experiments

https://gerrit.wikimedia.org/r/1152253

Mentioned in SAL (#wikimedia-operations) [2025-06-02T07:22:40Z] <phuedx@deploy1003> Started scap sync-world: Backport for [[gerrit:1152253|Beta Cluster: Support A/B experiments (T393918)]]

Mentioned in SAL (#wikimedia-operations) [2025-06-02T07:38:37Z] <phuedx@deploy1003> phuedx, dr0ptp4kt: Backport for [[gerrit:1152253|Beta Cluster: Support A/B experiments (T393918)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-06-02T07:58:40Z] <phuedx@deploy1003> Finished scap sync-world: Backport for [[gerrit:1152253|Beta Cluster: Support A/B experiments (T393918)]] (duration: 35m 59s)