This task is about implementing the changes necessary to ensure people are being bucketed in the Edit Check A/B test (T342930) as expected.

Bucketing requirements

• Bucketing should include both registered and unregistered users
• Everyone who is accessing a desktop or mobile main namespace page (NS:0) at any of the participating wikis, logged in and logged out, should have a 50% chance of being included/bucketed into the A/B test's control or test group
• Bucketing should be done on a per-Wikipedia basis. 50% of people within a given wiki should be placed within the control group; 50% should be bucketed in the test group
• The test group should have the edit check (references) with the edit reference reliability check. Note the inclusion of reference reliability is still pending the finalization of the design (T347531) and instrumentation (T352133).
• People should remain in the same group for the duration of the test.
• Testing should be done across all desktop skins where available

Instrumentation-Related Requirements

• A bucket is applied to these events so we can distinguish all events logged for the control group and the test group within the A/B test.
• An anonymous_user_token is applied to these events so we can track distinct unregistered users in addition to distinct registered users.

Participating wikis
Defined in T345298 (still being finalized)

Open questions

• 1. Will there be a third test group that is shown Edit Check with the Reference Reliability experience T276857 will implement?

DECIDED. Instead of completing a multivariant test, we are going to proceed with just two groups. The test group will have edit check (references) with the edit check reference reliability included. Rationale: We do not anticipate the inclusion of reference reliability will have a significant impact and are primarily interested in exploring usage to answer a few curiosities.

• 2. How might we ensure that the people who are included in the A/B test are the same people for whom volunteers have made Edit Check available by way of community configuration
  • Note: if who Edit Check is available to is NOT consistent across projects, we will not be able to draw cross-wiki conclusions.

Per team discussions, we don’t currently have a way to override the on-wiki config for the A/B test; however, it would be useful if the default settings were not radically changed in the AB test so we can be more confident on reporting the impact caused by Edit Check across all wikis. I've made a note in T345298#9401401 to try to select wikis that are comfortable with the default settings.

Also, the config is stored on-wiki with a revision history, so if we need to we can check what it was on any wiki at any point during the a/b test.

Done

• A patch has been deployed that meets what's described requirements sections above