Page MenuHomePhabricator

Understanding first day: post-deployment QA
Closed, ResolvedPublic

Description

This task is mostly for @nettrom_WMF.

Once the "EditorJourney" schema is deployed to Czech and Korean Wikipedias, we should quickly check that the data is flowing in in the format and volume we expected. We don't want weeks to go by before discovering that we can't use the data we collected.

Here are some ideas of what we should check (@nettrom_WMF or @Etonkovidova please add more):

  • Are there events from all new accounts created in Czech and Korean Wikipedias, as defined by accounts in the user table? Every account should have at least one event: the page it landed on after account creation.
  • Are any events from EditorJourney taking place more than 24 hours after account creation?
  • Are the URLs for the right namespaces being obscured?
  • Are the updated URLs for welcome templates in Czech and Korean Wikipedias recording correctly per T206882?
  • Is the new EditAttemptStep schema recording data for 100% of new registrants, but not for 100% of existing accounts?
  • Are we able to join to other important schemas, like ServerSideAccountCreation and EchoInteraction?
  • Is the new VisualEditorFeatureUse schema recording events for 100% of new registrants? Does it contain the new events we specified that we needed in T207802?

Event Timeline

Restricted Application added a subscriber: revi. · View Herald TranscriptNov 7 2018, 11:34 PM
MMiller_WMF moved this task from Inbox to Upcoming Work on the Growth-Team board.Nov 7 2018, 11:34 PM
MMiller_WMF added a project: Product-Analytics.
MMiller_WMF updated the task description. (Show Details)Nov 8 2018, 12:09 AM
nettrom_WMF moved this task from Triage to Next Up on the Product-Analytics board.Nov 9 2018, 7:37 PM
MMiller_WMF edited projects, added Growth-Team (Current Sprint); removed Growth-Team.
MMiller_WMF moved this task from Incoming to In Progress on the Growth-Team (Current Sprint) board.

Since we deployed today, this task is now in progress.

nettrom_WMF closed this task as Resolved.Jan 15 2019, 12:34 AM

We've QA'ed as much as we can, closing this for now.