Page MenuHomePhabricator

Generate Session Length test data
Open, MediumPublic

Description

We are looking for a way to QA the session tick instrumentation. We will do this by verifying if the time we actively spent on a wiki project is recorded correctly in the data lake. For this we need to be able to identify the user who creates some test data.

Data engineering team will
(1) create a session-tick-qa-testers user group on Test Wikipedia containing the user IDs of the expected QA tester(s) (in this case @Mayakp.wiki );
(2) update the session_tick instrument to inject qa: 1 into the test field if the current user is in the session-tick-qa-testers group. Then we can query the data lake for session tick events WHERE test.qa = 1.

Event Timeline

Change 668716 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[operations/mediawiki-config@master] WikimediaEvents: Create data QA group/right on testwiki

https://gerrit.wikimedia.org/r/668716

Change 668719 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/WikimediaEvents@master] Session tick: Add data QA flag if the user is in group data-qa

https://gerrit.wikimedia.org/r/668719

OK, the code for this is teed up. The MW config change shouldn't be merged/deployed until next week because of the no-Friday-deploys rule, but the WikimediaEvents change can be merged anytime. Once the config change lands, I'll add @Mayakp.wiki to the data-qa group on testwiki (and also myself, to ensure that it works first) using createAndPromote.php.

The data QA code should be removed from the instrument once data QA is complete, but I think it would be useful to keep the data-qa group in place on TestWiki even after this task is complete for future data QA needs.

Change 668716 merged by jenkins-bot:
[operations/mediawiki-config@master] WikimediaEvents: Create data QA group/right on testwiki

https://gerrit.wikimedia.org/r/668716

Mentioned in SAL (#wikimedia-operations) [2021-03-08T21:42:48Z] <mholloway-shell@deploy1002> Synchronized wmf-config/CommonSettings.php: WikimediaEvents: Create data QA group/right on testwiki (T276515) (duration: 00m 57s)

@Mayakp.wiki You should now be a member of the data-qa group on Test Wikipedia. You can verify this by viewing your current groups here: https://test.wikipedia.org/w/api.php?action=query&meta=userinfo&uiprop=groups

Please note that, in contrast to most event types, the session length instrument is active anytime you have an active session on the wikis, even without any action by you. The test.qa = 1 artifact will be emitted for session tick events generated on Test Wikipedia beginning as soon as your browser receives the updated script following tomorrow's planned MediaWiki deployment.

Please note that it may take some time for your browser to receive the updated script if an older version has not yet aged out of our edge caching infrastructure. I will follow up after tomorrow's deployment to confirm that the updated version of the instrument is being sent to clients. Also please note that the tick count is shared among browsers and across sites using cookies, so that ticks that are part of the same session but emitted when you are active on a wiki other than Test Wikipedia will not contain the test.qa = 1 artifact.

I am currently also a member of the data-qa group on Test Wikipedia but will plan to revoke my own membership momentarily to avoid polluting your testing data.

Happy testing!

Noted ! Thanks so much @Mholloway :)
how does this instrument behave with ad-blockers on ? (I have it switched off for testwiki, just asking for information)

@Mayakp.wiki, I've added some general info on ad blocker effects on event logging on-wiki at https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging#Ad_blockers. In a nutshell, if your ad blocker uses the EasyPrivacy block list, then requests to our analytics event intake services will be blocked. There may be others that we don't yet know about.

Which ad blocker are you using? If it's not already mentioned specifically, I would love to add it to the chart.

Change 670115 had a related patch set uploaded (by Brennen Bearnes; owner: Michael Holloway):
[mediawiki/extensions/WikimediaEvents@wmf/1.36.0-wmf.34] Session tick: Add data QA flag if the user is in group data-qa

https://gerrit.wikimedia.org/r/670115

Change 670115 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.36.0-wmf.34] Session tick: Add data QA flag if the user is in group data-qa

https://gerrit.wikimedia.org/r/670115

Change 668719 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Session tick: Add data QA flag if the user is in group data-qa

https://gerrit.wikimedia.org/r/668719

Which ad blocker are you using? If it's not already mentioned specifically, I would love to add it to the chart.

I have 'AdBlock' installed and I believe it is on the EasyPrivacy block list

Change 670496 had a related patch set uploaded (by Mholloway; owner: Jason Linehan):
[operations/mediawiki-config@master] Enable session tick on testwiki with 100% sampling

https://gerrit.wikimedia.org/r/670496

Change 670496 abandoned by Jason Linehan:
[operations/mediawiki-config@master] Enable session tick on testwiki with 100% sampling

Reason:
Went with https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventLogging/ /670598 as an alternative work-around

https://gerrit.wikimedia.org/r/670496

Mentioned in SAL (#wikimedia-operations) [2021-03-10T23:49:56Z] <mholloway-shell@deploy1002> Synchronized php-1.36.0-wmf.34/extensions/EventLogging: EventLogging: Stream always in sample if the user is in debugMode (T276515) (duration: 01m 23s)

mpopov triaged this task as Medium priority.Mar 15 2021, 4:09 PM
mpopov edited projects, added Product-Analytics; removed Product-Analytics (Kanban).
mpopov moved this task from Triage to Tracking on the Product-Analytics board.
mpopov added a subscriber: mpopov.

Thanks, @Mholloway! Do you want a separate task to remove this or re-use this for the clean-up?

Since it's a near-term follow-up, my vote would be to keep this one open.