Page MenuHomePhabricator

Issues with A/B test bucketing for language selector
Closed, DeclinedPublic

Description

Originally reported by Nirzar.

Visit https://hi.m.wikipedia.org/wiki/%E0%A4%AC%E0%A4%BF%E0%A4%B2%E0%A5%8D%E0%A4%B2%E0%A5%80?mobileaction=beta#/languages and you'll see the old language overlay.

The configuration says that the structured language overlay needs to load:
https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings.php#L13934

The code says that too: https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings.php#L13934

Running the following code in the console results in "A", which is what is expected:

mw.experiments.getBucket( mw.config.get( 'wgMFExperiments' ).languageOverlayBeta, mw.user.sessionId() )

So the bucket is A, and A bucket must see the structured overlay, but somehow it doesn't.

Event Timeline

just talked to @Jdlrobson

is this enabled on all wikipedias? we have to enable this feature on all wikipedia betas because this is a language feature and needs to work across all languages otherwise it's a broken experience.

Yes it is. I think the 'default' key in the config means 'all'.

The session id only refers to the current domain.

This means running an A/B test will need some more consideration. You'll need to pass the language overlay mode in the URL and pass the session over to the new domain.

Buckets are not sticky across sites.

Jdlrobson renamed this task from Wrong version of LanguageOverlay is showing in hiwiki to Issues with A/B test bucketing for language selector.Feb 26 2016, 10:31 PM

Same issue on Japanese, French and Portuguese
But not Russian. Weird.

Now PT and Hindi work fine for me. French seems to still show the old overlay.

Jdlrobson added a subscriber: dr0ptp4kt.

So should this task be declined @dr0ptp4kt ?
Are we planning to do anything / do we need to do anything here?