Page MenuHomePhabricator

Data QA for mediawiki_client_session_tick
Closed, ResolvedPublic

Description

Instrumentation data checks on the raw table when we deploy on group 2 wikis

Related Objects

Event Timeline

Mayakp.wiki triaged this task as High priority.EditedMar 10 2021, 10:18 PM
Mayakp.wiki moved this task from QA/Review to Sign-off on the Better Use Of Data board.

Completed basic data checks. Notebook uploaded to Github.
Pending: Session Length Instrumentation check - waiting on 100% sampling on testwiki

Mayakp.wiki moved this task from Sign-off to QA/Review on the Better Use Of Data board.
Mayakp.wiki added a subscriber: mpopov.

Re-assigning to @mpopov to complete the data QA.
Thanks a ton Mikhail !! :)

Queried Maya's test events from yesterday, posted them in the QA spreadsheet. I color-coded the three different sessions on the "QA Ticks" sheet and noted the issues on the "mediawiki_client_session_tick" sheet.

Observations:

  • Session A from Chrome (orange) had a lot of missing ticks. Highest one observed was 94 with a lot of ticks leading up to it missing. No 20s at all.
  • Session B from Firefox (blue) had the correct ticks (0, 1, 2)
  • Session C from Firefox (purple) went up to 32, confirming Maya's notes. However, the ticks started at 4. It seems like the instrument continued the session from before but skipped tick 3 (which it's not supposed to do), but what's troubling is that the session/counter did not reset like it was supposed to after more than 1 hour of absence.

Update: Some of the missing events and weirdness may have been from Maya accidentally opening other wikis during the test period. According to @Mholloway the session tick instrument is cross-wiki, so if she had any sessions open on other wikis and those tabs visible, then those events would have been sent from those non-test wikis and would not have been tagged as QA. We are unable to check with Maya whether that was the case and decided to do a complete re-test.

mpopov moved this task from QA/Review to Sign-off on the Better Use Of Data board.
mpopov added a subscriber: kzimmerman.

Performed a complete re-test of the instrument. Notebook uploaded to GitHub.

All checks passed!

Re-assigning to @kzimmerman for sign-off.

kzimmerman reassigned this task from kzimmerman to mpopov.