Instrumentation data checks on the raw table when we deploy on group 2 wikis
Description
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | mpopov | T275672 Data QA for mediawiki_client_session_tick | |||
Open | None | T276515 Generate Session Length test data |
Event Timeline
Completed basic data checks. Notebook uploaded to Github.
Pending: Session Length Instrumentation check - waiting on 100% sampling on testwiki
Queried Maya's test events from yesterday, posted them in the QA spreadsheet. I color-coded the three different sessions on the "QA Ticks" sheet and noted the issues on the "mediawiki_client_session_tick" sheet.
Observations:
- Session A from Chrome (orange) had a lot of missing ticks. Highest one observed was 94 with a lot of ticks leading up to it missing. No 20s at all.
- Session B from Firefox (blue) had the correct ticks (0, 1, 2)
- Session C from Firefox (purple) went up to 32, confirming Maya's notes. However, the ticks started at 4. It seems like the instrument continued the session from before but skipped tick 3 (which it's not supposed to do), but what's troubling is that the session/counter did not reset like it was supposed to after more than 1 hour of absence.
Update: Some of the missing events and weirdness may have been from Maya accidentally opening other wikis during the test period. According to @Mholloway the session tick instrument is cross-wiki, so if she had any sessions open on other wikis and those tabs visible, then those events would have been sent from those non-test wikis and would not have been tagged as QA. We are unable to check with Maya whether that was the case and decided to do a complete re-test.
Performed a complete re-test of the instrument. Notebook uploaded to GitHub.
All checks passed!
Re-assigning to @kzimmerman for sign-off.