Page MenuHomePhabricator

Post-deployment data QA
Closed, ResolvedPublic

Description

T245794 will make the V1.0 of the new Replying workflow available to contributors in production.

This task is about making sure the events we instrumented as part of T243364 are ending up in the database in the ways we expect.

Timing

This task can now be worked considering T245794 is resolved. Read: the feature is now available on our partner wikis as a Beta Feature.

Done

  • Document whether the events defined in the instrumentation spec [1] are landing the database in the ways we expect

  1. Google Doc (no sign in required): https://docs.google.com/spreadsheets/d/1iMWKmT7nHimjaa7OItGINoTVGtnaIy86I3ICj0RN5KY/edit#gid=1016912708

Event Timeline

ppelberg created this task.Mar 7 2020, 2:00 AM
ppelberg updated the task description. (Show Details)

Meeting notes: 11-March
Notes from Maya's and my conversations today:

  • We are planning to use the same queries to complete this task as we did for T243364 with the following changes:
    • Whether a person has turned on DiscussionTools in Beta Features

...As discussed during today's stand-up, v1.0 of the new Replying feature should be deployed:

  • As a Beta Feature
  • This upcoming Monday, 16-March
  • To the following Wikipedias: Arabic, Dutch, French and Hungarian

...the task description has been updated to include this information.

ppelberg set Due Date to Mar 27 2020, 7:00 AM.Mar 12 2020, 8:56 PM
kzimmerman triaged this task as Medium priority.Mar 17 2020, 5:32 PM
kzimmerman added a subscriber: kzimmerman.

@ppelberg I understand this may be delayed due to delayed deployments - can you update the due date & other information to reflect that?

ppelberg updated the task description. (Show Details)Mar 17 2020, 8:15 PM
ppelberg removed Due Date.

@ppelberg I understand this may be delayed due to delayed deployments - can you update the due date & other information to reflect that?

Absolutely – I've made the following changes (below) to this task. Thank you for the ping, Kate.

Task changes:

  • REMOVED: task due date
  • ADDED: deployment timing to task description

Thank you! I'll move it to our backlog for now, then.

Update: 26-March
Deployment is scheduled for this upcoming Tuesday, 31-March, assuming no critical issues surface in QA which will finish Monday, 30-March.

ppelberg updated the task description. (Show Details)Apr 1 2020, 7:23 PM

Updating task description to reflect yesterday's deployment and resolution of T245794, the task that had been blocking work on this one.

Increase Sampling Rate of DiscussionTools edits :

While doing the data QA we have observed that we are getting very few ‘discussion tools’ related events in the EditAttemptStep schema. This is possibly because the Replying V1.0 feature is deployed on 4 small wikis - nl, hu, ar, fr as an opt-in feature and the schema is sampled at 1/16 rate.
These are all the events on each wiki from Mar 31 to Apr 10.

My request is to oversample all DiscussionTools events to 1/5.

**Note: Given the impact, I believe that this should not cause a huge spike in eventlogging. However if we need to increase or decrease the sampling we can do so by changing the config override T250086

Next steps
Next steps from the conversations Maya and I had today:

  • @Mayakp.wiki to provide update about how the deployment of T250086 has affected the number of events being logged in the DB.
Mayakp.wiki closed this task as Resolved.Apr 30 2020, 9:52 PM

QC Document link Data looks correct for the most part.

  • Outputs and Screenshots are provided in a separate tab for reference.
  • Note: I did expect a big rise in the number of events after the sampling rate was changed to 1/5. However, that doesn't seem to be the case. With that said, it does seem sufficient to do our analysis. I will monitor the events even after this task is closed.