Page MenuHomePhabricator
Create Task
Maniphest T206543

Make it possible to conditionally oversample Edit events
Closed, ResolvedPublic
Actions

Description

In mediawiki/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.trackSubscriber.js, mediawiki/extensions/WikiEditor/includes/WikiEditorHooks.php, and mediawiki/extensions/WikiEditor/modules/ext.wikiEditor.js there is code that samples 6.25% of interactions for logging data to the Edit schema.

For our work in understanding first day, we want to be able to sample at 100%. Probably the easiest way is to use a configuration variable we'll be using elsewhere ($wgWMEUnderstandingFirstDay), and if that is set to true, send all transactions instead of just 6.25%.

Details

SubjectRepoBranchLines +/-
Oversample Schema:Edit events when configured to do somediawiki/extensions/WikiEditormaster+16 -9
Use Schema:Edit sampling rate config from WikimediaEventsmediawiki/extensions/WikiEditormaster+18 -13
Add a server-side sessionInSample() functionmediawiki/extensions/EventLoggingmaster+11 -0
Oversample Schema:Edit events when configured to do somediawiki/extensions/MobileFrontendmaster+5 -2
Oversample Schema:Edit events when configured to do somediawiki/extensions/VisualEditormaster+2 -1
Add oversampling support for Schema:Editmediawiki/extensions/WikimediaEventsmaster+8 -1
Use Schema:Edit sampling rate config from WikimediaEventsmediawiki/extensions/VisualEditormaster+18 -7
Use Schema:Edit sampling rate from WikimediaEventsmediawiki/extensions/MobileFrontendmaster+1 -3
Make sampling rate for Schema:Edit configurablemediawiki/extensions/WikimediaEventsmaster+5 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

kostajh created this task.Oct 9 2018, 3:15 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 9 2018, 3:15 PM
kostajh mentioned this in T205762: Understanding first day: determine which instrumentation can be re-used.Oct 9 2018, 3:44 PM
kostajh edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.Oct 10 2018, 6:07 PM
kostajh moved this task from Incoming to Ready for Development on the Growth-Team (Sprint 0 (Growth Team)) board.
SBisson subscribed.Oct 10 2018, 6:43 PM

When $wgWMEUnderstandingFirstDay is true, do we want to capture 100% of traffic or accounts that are less than 24h old?

kostajh added a comment.Oct 10 2018, 6:48 PM
In T206543#4656081, @SBisson wrote:

When $wgWMEUnderstandingFirstDay is true, do we want to capture 100% of traffic or accounts that are less than 24h old?

If both $wgWMEUnderstandingFirstDay and PageViews::userIsInCohort( $user ) are true, then we should log 100% of events. Otherwise we should stick with the 6.25% sampling.

MMiller_WMF added a parent task: T205762: Understanding first day: determine which instrumentation can be re-used.Oct 11 2018, 3:46 PM
MMiller_WMF subscribed.
Catrope claimed this task.Oct 15 2018, 9:00 PM
Catrope moved this task from Ready for Development to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.
gerritbot subscribed.Oct 15 2018, 10:55 PM

Change 467542 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/WikimediaEvents@master] Make sampling rate for Schema:Edit configurable

https://gerrit.wikimedia.org/r/467542

gerritbot added a project: Patch-For-Review.Oct 15 2018, 10:55 PM

Change 467543 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/VisualEditor@master] Use Schema:Edit sampling rate config from WikimediaEvents

https://gerrit.wikimedia.org/r/467543

gerritbot added a comment.Oct 15 2018, 10:55 PM

Change 467544 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/WikiEditor@master] Use Schema:Edit sampling rate config from WikimediaEvents

https://gerrit.wikimedia.org/r/467544

nshahquinn-wmf added projects: VisualEditor, Product-Analytics.Oct 15 2018, 10:55 PM
nshahquinn-wmf renamed this task from Add configuration variable for sampling rate on Edit schema to Make it possible to conditionally oversample Edit events.Oct 16 2018, 1:24 AM
nshahquinn-wmf subscribed.

@Catrope's current plan:

  • Migrate VE and WikiEditor to the mw.eventLog.Schema class, which was created after the VE instrumentation code was written
  • Make the sampling rate configurable (like in this WIP change, but as an actual number rather than a weird hacky hex thing)
  • Add oversampling support to mw.eventLog.Schema, which adds isOversampled=true to the event data for oversampled events (i.e. events that bypassed sampling and would not have been logged otherwise)
  • Add code to WikimediaEvents that triggers oversampling in the cases where we need it
• Deskana moved this task from To Triage to External and Administrivia on the VisualEditor board.Oct 16 2018, 11:18 AM
• Deskana added subscribers: DLynch, • Deskana.

This is great. Thank you!

I've moved this task to "external" to reflect that another team is actively working on it, but I do consider this a shared responsibility, so let me know if you need help or Editing engineering time.

Catrope added a subscriber: Krinkle.Oct 16 2018, 7:25 PM
In T206543#4668988, @Neil_P._Quinn_WMF wrote:

@Catrope's current plan:

  • Migrate VE and WikiEditor to the mw.eventLog.Schema class, which was created after the VE instrumentation code was written
  • Make the sampling rate configurable (like in this WIP change, but as an actual number rather than a weird hacky hex thing)
  • Add oversampling support to mw.eventLog.Schema, which adds isOversampled=true to the event data for oversampled events (i.e. events that bypassed sampling and would not have been logged otherwise)
  • Add code to WikimediaEvents that triggers oversampling in the cases where we need it

@Krinkle tells me that he plans on removing mw.eventLog.Schema, so instead of using that, I will standardize on using mw.eventLog.inSample() for sampling, centralize the sampling rate config variable (based on what MobileFrontend does currently), and add a config var to trigger oversampling.

gerritbot added a comment.Oct 17 2018, 3:03 AM

Change 467862 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/EventLogging@master] Add a server-side randomTokenMatch() function

https://gerrit.wikimedia.org/r/467862

gerritbot added a comment.Oct 17 2018, 3:07 AM

Change 467863 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/MobileFrontend@master] Use Schema:Edit sample rate from WikimediaEvents

https://gerrit.wikimedia.org/r/467863

gerritbot added a comment.Oct 17 2018, 5:47 AM

Change 467884 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/WikimediaEvents@master] Add oversampling support for Schema:Edit

https://gerrit.wikimedia.org/r/467884

gerritbot added a comment.Oct 17 2018, 5:47 AM

Change 467885 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/WikiEditor@master] Oversample Schema:Edit events when configured to so

https://gerrit.wikimedia.org/r/467885

gerritbot added a comment.Oct 17 2018, 5:47 AM

Change 467886 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/MobileFrontend@master] Oversample Schema:Edit events when configured to so

https://gerrit.wikimedia.org/r/467886

gerritbot added a comment.Oct 17 2018, 5:48 AM

Change 467887 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/VisualEditor@master] Oversample Schema:Edit events when configured to so

https://gerrit.wikimedia.org/r/467887

Catrope added a comment.Oct 17 2018, 6:09 AM

In addition to the 9 (sorry!) patches tagged with this task, I've also amended https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/464432 (for T205759) to oversample Edit events in the relevant cases. I'm now done with the implementation and ready for initial review, but I haven't yet put together all these 10 patches to test that oversampling actually works in all 3 of the extensions that emit Edit events. I'll do that tomorrow, then move this task to the code review column.

gerritbot added a comment.Oct 17 2018, 10:23 AM

Change 467542 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Make sampling rate for Schema:Edit configurable

https://gerrit.wikimedia.org/r/467542

gerritbot added a comment.Oct 17 2018, 10:33 AM

Change 467863 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Use Schema:Edit sampling rate from WikimediaEvents

https://gerrit.wikimedia.org/r/467863

ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.1; 2018-10-23).Oct 17 2018, 11:00 AM
nshahquinn-wmf moved this task from Triage to Tracking on the Product-Analytics board.Oct 18 2018, 6:52 PM
Catrope moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 18 2018, 8:01 PM

I have now tested these patches and confirmed that they correctly oversample new users and mark oversampled events as such. They're ready for code review.

gerritbot added a comment.Oct 18 2018, 9:54 PM

Change 467543 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Use Schema:Edit sampling rate config from WikimediaEvents

https://gerrit.wikimedia.org/r/467543

gerritbot added a comment.Oct 23 2018, 8:05 PM

Change 467884 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Add oversampling support for Schema:Edit

https://gerrit.wikimedia.org/r/467884

ReleaseTaggerBot edited projects, added MW-1.33-notes (1.33.0-wmf.2; 2018-10-30); removed MW-1.33-notes (1.33.0-wmf.1; 2018-10-23).Oct 23 2018, 9:01 PM
gerritbot added a comment.Oct 24 2018, 9:21 PM

Change 467887 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Oversample Schema:Edit events when configured to do so

https://gerrit.wikimedia.org/r/467887

gerritbot added a comment.Oct 24 2018, 9:26 PM

Change 467886 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Oversample Schema:Edit events when configured to do so

https://gerrit.wikimedia.org/r/467886

gerritbot added a comment.Oct 24 2018, 9:26 PM

Change 467862 merged by jenkins-bot:
[mediawiki/extensions/EventLogging@master] Add a server-side sessionInSample() function

https://gerrit.wikimedia.org/r/467862

gerritbot added a comment.Oct 24 2018, 9:37 PM

Change 467544 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] Use Schema:Edit sampling rate config from WikimediaEvents

https://gerrit.wikimedia.org/r/467544

gerritbot added a comment.Oct 24 2018, 9:38 PM

Change 467885 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] Oversample Schema:Edit events when configured to do so

https://gerrit.wikimedia.org/r/467885

Jdforrester-WMF removed a project: Patch-For-Review.Oct 24 2018, 9:40 PM
kostajh moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 24 2018, 9:57 PM

All the patches are merged, ready for QA!

Etonkovidova closed this task as Resolved.Nov 1 2018, 6:29 PM
Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptNov 1 2018, 6:29 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits