Page MenuHomePhabricator

Understanding first day: post-deployment QA
Closed, ResolvedPublic


This task is mostly for @nettrom_WMF.

Once the "EditorJourney" schema is deployed to Czech and Korean Wikipedias, we should quickly check that the data is flowing in in the format and volume we expected. We don't want weeks to go by before discovering that we can't use the data we collected.

Here are some ideas of what we should check (@nettrom_WMF or @Etonkovidova please add more):

  • Are there events from all new accounts created in Czech and Korean Wikipedias, as defined by accounts in the user table? Every account should have at least one event: the page it landed on after account creation.
  • Are any events from EditorJourney taking place more than 24 hours after account creation?
  • Are the URLs for the right namespaces being obscured?
  • Are the updated URLs for welcome templates in Czech and Korean Wikipedias recording correctly per T206882?
  • Is the new EditAttemptStep schema recording data for 100% of new registrants, but not for 100% of existing accounts?
  • Are we able to join to other important schemas, like ServerSideAccountCreation and EchoInteraction?
  • Is the new VisualEditorFeatureUse schema recording events for 100% of new registrants? Does it contain the new events we specified that we needed in T207802?

Event Timeline

MMiller_WMF edited projects, added Growth-Team (Current Sprint); removed Growth-Team.
MMiller_WMF moved this task from Incoming to In Progress on the Growth-Team (Current Sprint) board.

Since we deployed today, this task is now in progress.

We've QA'ed as much as we can, closing this for now.