Page MenuHomePhabricator

Create API for setting 'bucket' in VE analytics
Open, Needs TriagePublic

Description

This probably will require duplicating code in VE and MF.

It should be possible to set multiple buckets, which are sent as a comma separated list according to the schema (https://meta.wikimedia.org/wiki/Schema:EditAttemptStep)

Event Timeline

Esanders created this task.Mar 22 2019, 6:17 PM
JTannerWMF moved this task from To Triage to FY 19-20 on the VisualEditor board.Mar 26 2019, 3:18 PM

For future reference it should be noted that userId's are not persistent across wikis for the same user, so if we have a more complex bucket function we should use username to ensure users end up in the same bucket across wikis.

Good point! I documented this at a page I just created: wikitech:A/B testing. Feel free to dump other thoughts and links there too.

@Jdforrester-WMF suggested we could expose the user's global ID from CentralAuth to the client and use that instead.

https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/CentralAuth/+/500826 provides that client-side, but I think the Performance team should weigh in before merging.

The would allow %2 bucketing in the client, which has the advantage of being fast.

Hashing based on user name (which is globally invariant) and an experiment seed is slower and more complex in the client, with these (probably minor) benefits:

  • Mixing groups across experiments
  • No time-based grouping from alternating (important of small wikis with small samples)