Page MenuHomePhabricator
Create Task
Maniphest T128214

Issues with A/B test bucketing for language selector
Closed, DeclinedPublic
Actions

Description

Originally reported by Nirzar.

Visit https://hi.m.wikipedia.org/wiki/%E0%A4%AC%E0%A4%BF%E0%A4%B2%E0%A5%8D%E0%A4%B2%E0%A5%80?mobileaction=beta#/languages and you'll see the old language overlay.

The configuration says that the structured language overlay needs to load:
https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings.php#L13934

The code says that too: https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings.php#L13934

Running the following code in the console results in "A", which is what is expected:

mw.experiments.getBucket( mw.config.get( 'wgMFExperiments' ).languageOverlayBeta, mw.user.sessionId() )

So the bucket is A, and A bucket must see the structured overlay, but somehow it doesn't.

Related Objects
Search...

Event Timeline

• bmansurov created this task.Feb 26 2016, 9:28 PM
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 26 2016, 9:28 PM
• Nirzar added a subscriber: Jdlrobson.Feb 26 2016, 9:50 PM

just talked to @Jdlrobson

is this enabled on all wikipedias? we have to enable this feature on all wikipedia betas because this is a language feature and needs to work across all languages otherwise it's a broken experience.

• Nirzar triaged this task as High priority.Feb 26 2016, 9:50 PM
• bmansurov added a comment.Feb 26 2016, 9:55 PM

Yes it is. I think the 'default' key in the config means 'all'.

• Nirzar added a comment.Feb 26 2016, 9:58 PM

@bmansurov oh okay. so its a bug!

Jdlrobson added a comment.Feb 26 2016, 10:28 PM

The session id only refers to the current domain.

This means running an A/B test will need some more consideration. You'll need to pass the language overlay mode in the URL and pass the session over to the new domain.

Buckets are not sticky across sites.

Jdlrobson renamed this task from Wrong version of LanguageOverlay is showing in hiwiki to Issues with A/B test bucketing for language selector.Feb 26 2016, 10:31 PM
Jdlrobson added a parent task: T127212: Remove sampling rate artifacts for language switcher after enough data collected.
Jdlrobson added a comment.Feb 26 2016, 10:50 PM

Same issue on Japanese, French and Portuguese
But not Russian. Weird.

Jdlrobson added a comment.Feb 26 2016, 10:59 PM

Now PT and Hindi work fine for me. French seems to still show the old overlay.

• bmansurov mentioned this in T128917: Head-to-head A/B test language switcher alternative.Mar 8 2016, 12:01 PM
Jdlrobson assigned this task to dr0ptp4kt.Mar 16 2016, 12:24 AM
Jdlrobson added a subscriber: dr0ptp4kt.

So should this task be declined @dr0ptp4kt ?
Are we planning to do anything / do we need to do anything here?

dr0ptp4kt closed this task as Declined.Mar 16 2016, 4:03 PM

Declined.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits