Page MenuHomePhabricator

Synthetic experiment to test new event path (round 2)
Closed, ResolvedPublic1 Estimated Story Points

Description

Following up on T415708: Synthetic experiment to test new event path and T415790#11604075:

RECOMMENDATION: An experiment where we test both external paths using navigator.sendBeacon() directly AND also use mw.tk.getInstrument().submitInteraction() to evaluate how much event loss all the stuff in the middle is responsible for.

This work builds on externalPathTest.js (instrument config, experiment config)

We will keep the configurations mostly the same and only need to adjust the start & end dates of both the experiment and the instrument.

Acceptance criteria

  • In addition to the current methods for sending events, the event is sent to the new external path (/ins-502b/v2/events) directly via navigator.sendBeacon(), for 3 events total per page visit for users enrolled in the experiment
  • All 3 events include UA strings
    • synth-test-external-path configured to collect agent_ua_string contextual attribute
    • Events sent via sendBeacon() (directly) manually populate agent.ua_string property in the event data
  • action_context includes JSON-stringified object with properties:
    • tk_sdk (boolean) true for event sent via submitInteraction() and false for other two events sent via sendBeacon directly
    • ext_path (string) "old" or "new"
    • tz_offset (int) output of getTimezoneOffset(), which we will use to study how the class of device used varies by local (not UTC) time of day
  • Data collection is active for 7 days

Event Timeline

KReid-WMF set the point value for this task to 1.

Change #1240405 had a related patch set uploaded (by Kareid; author: Kareid):

[mediawiki/extensions/WikimediaEvents@master] Add third event for synth-test-external-path experiment

https://gerrit.wikimedia.org/r/1240405

Change #1241869 had a related patch set uploaded (by Kareid; author: Kareid):

[integration/config@master] Add TestKitchen dependency to WikimediaEvents

https://gerrit.wikimedia.org/r/1241869

Change #1241869 merged by jenkins-bot:

[integration/config@master] Add TestKitchen dependency to WikimediaEvents

https://gerrit.wikimedia.org/r/1241869

Have deployed the CI config change Add TestKitchen dependency to WikimediaEvents. Please make sure that WikimediaEvents tests that depends on TestKichen are skipped when TestKichen is not present.

The reason is that in CI WikimediaEvents is a dependency of the following repositories:

AbuseFilter, AntiSpoof, CentralAuth, CheckUser, cldr, ConfirmEdit, Echo, EmailAuth, EventBus, EventLogging, EventStreamConfig, FlaggedRevs, GlobalBlocking, GlobalPreferences, IPReputation, LoginNotify, OATHAuth

And because CI runs every tests, for those repositories it will run all of WikimediaEvents tests and the one relying on TestKitchen will fail.

Edit: we can follow up on the code review happening at https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/1240405/comment/adb9ae9a_d73dd65d/

phuedx updated the task description. (Show Details)

Change #1240405 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Add third event for synth-test-external-path experiment

https://gerrit.wikimedia.org/r/1240405