Page MenuHomePhabricator

Create harness for A/B test for logged in users
Open, HighPublic3 Estimated Story Points

Description

Spun out of T275807

Readers Web will be A/B testing the existing treatment of the language switcher and a new treatment being worked on as part of Desktop Improvements. The initial cohort will be for logged-in users only

AC

  • We should bucketed on a user's centralized ID (global ID) so that they remain consistently bucketed across sites (e.g. If I switch from English to French I should be bucketed the same)
  • When I visit a wiki with the A/B test enabled, I receive either the existing treatment or the new one
  • When I visit a wiki with the A/B test enabled and I have the magic query string parameter set then it should have the following effects:
ValueEffect
undefinedI'm entered into the A/B test and bucketed as usual
controlI see the existing treatment
AI see the new treatment

Developer Notes

  1. Bucketing should happen on the server side
    • Would adding a class to the body tag to help with bucketing
  2. We should bucketed on a user's centralized ID (global ID) so that they remain consistently bucketed across sites…

This change be achieved with the following:

$lookup = CentralIdLookup::factoryNonLocal();
$id = null;
if ( $lookup ) {
  $id = $lookup->centralIdFromLocalUser( $user );
}
// The central ID lookup failed?
if ( !$id ) {
  $id = $user->getId();
}

QA Steps

With Even Logged in User

  1. Visit https://patchdemo.wmflabs.org/wikis/51ef0c9126/wiki/Tree
  2. Login with username: alice and password patchdemo1
  3. In dev console, run mw.user.getId(); ensuring that user has an even user id.
  4. Assert that language button appears and languages do NOT appear in sidebar.
  5. Visit https://patchdemo.wmflabs.org/wikis/51ef0c9126/wiki/Tree?languageinheader=0
  6. Assert that languages appear in the sidebar and that the language button does NOT appear

With Odd Logged in User

  1. Visit https://patchdemo.wmflabs.org/wikis/51ef0c9126/wiki/Tree
  2. Login with username: bob and password patchdemo1
  3. In dev console, run mw.user.getId(); ensuring that user has an odd user id.
  4. Assert that languages appear in the sidebar and that the language button does NOT appear
  5. Visit https://patchdemo.wmflabs.org/wikis/51ef0c9126/wiki/Tree?languageinheader=1
  6. Assert that language button appears and languages do NOT appear in sidebar.

With Anonymous User (and languageinheader config off for anons)

  1. Logout and assert that you are an anonymous user.
  2. Visit https://patchdemo.wmflabs.org/wikis/51ef0c9126/wiki/Tree
  3. Assert that languages appear in the sidebar and that the language button does NOT appear

Event Timeline

Jdlrobson updated the task description. (Show Details)
phuedx updated the task description. (Show Details)

Change 683451 had a related patch set uploaded (by Nray; author: Nray):

[mediawiki/skins/Vector@master] Create A/B test harness for Language in header feature

https://gerrit.wikimedia.org/r/683451

nray removed nray as the assignee of this task.Thu, Apr 29, 12:57 AM
nray added a subscriber: nray.
nray removed nray as the assignee of this task.Tue, May 4, 2:32 AM
nray assigned this task to phuedx.
nray removed nray as the assignee of this task.Tue, May 4, 7:04 PM
nray assigned this task to phuedx.
nray removed phuedx as the assignee of this task.
nray assigned this task to phuedx.

Change 683451 merged by jenkins-bot:

[mediawiki/skins/Vector@master] Create A/B test harness for Language in header feature

https://gerrit.wikimedia.org/r/683451

nray claimed this task.
nray added a subscriber: Edtadros.

I need to add QA steps

Change 685944 had a related patch set uploaded (by Nray; author: Nray):

[mediawiki/skins/Vector@master] DNM: Config for QA of T280825 via PatchDemo

https://gerrit.wikimedia.org/r/685944

nray updated the task description. (Show Details)

Change 685944 abandoned by Nray:

[mediawiki/skins/Vector@master] DNM: Config for QA of T280825 via PatchDemo

Reason:

https://gerrit.wikimedia.org/r/685944