FWIW I don't think it's necessary to revert the patch that I merged today. @Jdlrobson left some helpful comments on the patch that should get us to the solution 🙂

Being bold. I re-ran the tests and it passes. I've also confirmed that fuzzy matches are being returned on the Beta Cluster:

Thanks for writing this up, @awight.

Could this be caused by the styles in rEPOP7b0937c5a865: Use CSS clip-path instead of SVG when supported. not being flipped by CSSJanus?

Nice work on the proof of concept!

In T275807#6864971, @Jdlrobson wrote:

Is old treatment languages in the sidebar? Is new treatment language in the header?

Readers Web will be A/B testing the existing treatment of the language switcher and a new treatment being worked on as part of Desktop Improvements. The initial cohort will be all users (not just logged-in users).

Note, given the old and new treatment require HTML changes I think an A/B test could only feasibly be run here for logged in users.

I'm actually going to move this one back into Needs Analysis. AFAICT @awight's approach in rEEVLaca489f139cd: User edit count bucketing is one @Krinkle advised against in T210106#5829916 so we should probably chat about it.

In T275766#6862608, @Jdlrobson wrote:

I think once T275766#6862597 is clarified (via an addition to acceptance criteria) this should be moved into upcoming for estimation.

If my understanding is correct, using time the page was open might include instances where the page is just open in the background during tabbed browsing. This could skew our results and understanding of how long a user is actually viewing the page before they switch languages.

In T210106#6862686, @awight wrote:

Sorry I wasn't aware of this task until now, and may have duplicated some work. In patch aca489f139cd13df, I added code to the EventLogging extension which makes wgUserEditCountBucket available from JS, and EventLoggingServices::getInstance()->getUserBucketProvider()->getUserEditCountBucket( $user ) from the backend.

Yes! Thanks for the ping, @Jdlrobson.

@Edtadros: If you have access to Hue (i.e. you've signed an NDA and you're in the nda LDAP group, then you should be able to test this. Alternatively, @MNeisler or I could.

Regrettably, this will have to tested by checking data in the event.searchsatisfaction Hive table once the change has gone live – it'll be deployed to the group0 wikis tomorrow – as there's no way to override the sampling code that enables the SearchSatisfaction instrument.

Excellent! I'd asked @Mholloway if he was looking for a merger just the other day. Great work, everyone!

From an engineering perspective only, this is roughly an S – it's a one-line config change and deployment. The config change is as follows:

I've scheduled the above for deployment.

In T272991#6780526, @phuedx wrote:

We might want to revisit reusing the user edit count bucketing code again. However, code reuse and testing in possible but non-trivial in the WikimediaEvents, which is geared towards isolated modules that do everything that they need.

Thanks, @nshahquinn-wmf!

In T268504#6821401, @MNeisler wrote:

Instrumentation: Initial and final language
At the time of writing, the UniversalLanguageSelector instrument sets the interfaceLanguage property to the final language for language-change events. However, for all other events, the interfaceLanguage property is set to the initial language.
We could infer the initial language from the domain of the intake URL, which is logged as the event.meta.domain field.

Could the initial language also be inferred from the event.ContentLanguage field for language-change events? Either way, I agree with the addition of the selectedLanguageproperty to clarify the final language selection for events.

/cc @ovasileva

In T274210#6817031, @Jdlrobson wrote:

If I'm reading correctly, it looks like InitialiseSettings-labs.php can override keys in InitialiseSettings.php provided the keys are prefixed with "-" is that right? That said I've made mistakes hereb efore.

So in the case of https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings-labs.php#L372-L384 the reason the logo doesn't apply to https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page?useskinversion=1 is because no entry for 'wikipedia' has been defined and wmgSiteLogo1x should be -wmgSiteLogo1x ... am I reading that correctly?

In T268504#6817487, @nray wrote:

In T268504#6796185, @phuedx wrote:

AC: Allow logged-in (logged-out users nice to have) to be bucketed into either the new treatment or the control

Logged-in and -out users

We can't bucket logged-out users on the server, which introduces a number of complications:

1. We'll have to deliver the page in an "undefined" state until we can make a determination. Since all JavaScript is loaded asynchronously so as not to block the initial rendering of the page, there'll be a small delay until either treatment is available to the user, i.e. there's a small risk that a user who has previously expanded the sidebar, might see a list of 300+ languages suddenly appearing – I've judged the risk as small because the sidebar is collapsed by default for anonymous users with fresh sessions (and sessions are highly likely to be refreshed after seven days).

If we do end up bucketing anon users though, would we use the same client-side code to bucket logged-in users? (where I think the sidebar is open by default, and I would think 300+ languages suddenly appearing in the sidebar would be more noticeable)?

In T268504#6817487, @nray wrote:

Thank you @phuedx for the great detailed notes!

In T268504#6810416, @phuedx wrote:

@alexhollender and I discussed this on Thursday and agreed that hiding the language list in the sidebar and hiding the language switcher until the user is bucketed is the simplest solution from both a design and engineering perspective.

Could you elaborate on how you envision this working for no-js users just so I'm on the same page? Were you envisioning the server would render the language list in the sidebar (the status quo), the language button in the body header but with an empty .vector-menu-content-list list, and we would use the client-js class to hide both of these before the JS executes? If the user is bucketed in the body header group on the client then ULS would use the list of languages in the sidebar for its data?

In T268504#6804647, @LGoto wrote:

Questions from Sam: Is it worth thinking about the affect on the a/b test? Is there a design treatment that is an ambiguous state we send to everyone?

In T272530#6796528, @Gilles wrote:

First of all, how dated are the versions of Vega we're using for this? Maybe newer ones are leaner or more modular.

Is the JS only required for interaction? I thought it was needed for rendering too. If only needed for interaction, then waiting for user interaction makes total sense.

Instrument: Clicks to language list in control group

Instrumentation: Time on page prior to switching to a different language

☝️ Per our discussion yesterday, we'll sign this off now and test the change after we've enabled the A/B test on the Beta Cluster.

Instrumentation: Initial and final language

Instrumentation: Number of new users accessing language switching functionality

Instrumentation: Clicks to new button

AC: Allow logged-in (logged-out users nice to have) to be bucketed into either the new treatment or the control