This task is about implementing the changes necessary to ensure the "right people" are are included in the Reply Tool A/B test in the "right way."
People who meet all of the "Conditions" listed below, ought to have a 50% chance of being bucketed into the A/B test's control or test group.
- Editing at the following Wikipedias (source: T267379):
- Have not used the Reply Tool before.
- In this context, "not used" is being defined as people whose discussiontools-editmode preference is empty.
- People who are logged in
Two additional notes:
- Bucketing ought to be done a per wiki basis.
- The software will need to remember that someone was included in the A/B test so they are not mistakenly removed from it.
- 1. Does this task need to be blocked on having defined the list of Wikipedias that will be participating in the test? See: T267379.
- Moot point as T267379 has been resolved.
- 2. What should happen when someone from the control group manually enables the Reply Tool in Special:Preferences? Should them changing their preference cause them to be added to the test group? Should their usage of the Reply Tool, that they would have manually enabled, be considered part of the control group?
- When someone from the control group manually enables the Reply Tool in Special:Preferences, they should remain in the control group. See: T268191#6746113.
- 3. Can the bucketing be deployed to test.wikipedia.org to conduct the QA that will happen in T268193. Additional context: T268191#6746113.
- We decided to use id.wiki for QA. See: T268191#6763627.
- The "Bucketing criteria" above have been implemented
Verify on Beta that the code required to assign people to the test and control groups is working as expected.
- This happened in: T268193.
Note: verification that people are being bucketed as expected will happen in T268193.