@Tbayer, presently the stable channel is at 1/1000 sampling. In order to observe the impact of positioning the language switcher button at the top of the article on a handful of wikis, do you recommend increasing the sampling rate? If so, do you mind if we increase the sampling rate on those wikis only? And at what rough sampling rates would you want for each of the three classes of wiki (small, medium, large) ?

@Jdlrobson I understand you have data about wiki sizes, can you recommend 2 small 2 medium and 1 large wiki to deploy this to that are different from the ones we're using for the performance work?

@JKatzWMF and @dr0ptp4kt to flesh this out and discuss technical blockers before the team reviews for estimation.

@dr0ptp4kt Need to define which wikis

i really would strongly advise deploying to all wikis.
Especially given T139794 Show users that language button has moved will behave differently on different wikis.

@Jdlrobson, suppose we picked only one wiki on which to do this instead of 5. What are the pros and cons of doing one wiki, followed by all VS. all wikis all at once?

One wiki

Pros:

• There is a small chance we might hit problems with cached HTML but imo these can be caught through vigorous testing before a launch.
• In the event of an issue any major problems can be contained to one wiki/rolled back

Cons:

• Inconsistent/confusing ux for multilingual users where they use one wiki where it is enabled and one wiki where it is not. We will be pointing to the new language button on one but they would have to use it below on the other
• Two SWAT deploys at least rather than 1

All wikis

Pros:

• Any bugs will bubble up quicker given more users will be exposed to feature and through Village pump/mailing list/Twitter
• Consistent UX across all wikis

Cons:

• If something catastrophic happened e.g. API requests melted the server - would be hard to recover from.

Assessment

My assessment of the risk of something bad happening are very low. Moving the language button to the top will generate in more clicks but will it lead to an overwhelming amount of API calls? I think not. I think the consistent UX across wikis is very important (but @Nirzar should decide about that).

Given we've been in beta for some time I'm really not concerned about minor issues. If we find them I'd argue we can squash them quicker with a follow up SWAT.

I think it is very important to go with All wikis option.

Inconsistent/confusing ux for multilingual users where they use one wiki where it is enabled and one wiki where it is not. We will be pointing to the new language button on one but they would have to use it below on the other

This feature is for multilingual users, we can't afford to have this negative point about small wikis.

@Nirzar, to clarify your point, doing this for one wiki for one week would be unacceptable. Do I have that right?

@dr0ptp4kt yes. since the user will switch language to other wiki and the button will be in different place without any understanding of why.

If that's the consensus I'm ok with it.

My personal opinion is that we should production-test this on a wiki given we can't reproduce locally the production infrastructure with the caching conditions.

As a note, rollback is not that easy/clean if the feature (which changes server html) has been deployed for some time. Then even if we revert the patches, there may be cached HTML pages that were generated by the initial deploy, but now will be acted on by the reverted resources. To properly roll back we would need to purge, which is not a good idea (but better on one than 25).

It's also not that big of a change, so I understand where the confidence is coming from.

As I said, fine with whatever, both pros and cons are valid on their own.

One compromise is 1 deployment on a Wednesday followed by all others on the Thursday. Really problems with cached pages should have been thought about and tested on the beta cluster (we can test it there if we plan to prior to merging)

Copying in verbiage from another task:

In T140260#2505821, @Jdlrobson wrote:

@Jhernandez we can verify the cached HTML issue locally. I just verified this works as expected myself.

The verification step is as follows:

• $wgMinervaUsePageActionBarV2 = false;
• Visit a page with languages as an anon.
• View source and save the file to the MobileFrontend directory as cached.html
• Enable $wgMinervaUsePageActionBarV2 = true
• Visit http://localhost:8888/w/extensions/MobileFrontend/cached.html

You can also do similar tests by checking out older revisions e.g.

• git checkout 70047ab0f74186e78bbdd69ac1cab12349c835c3
• Visit a page with languages as an anon.
• View source and save the file to the MobileFrontend directory as cached.html
• git checkout master
• Visit http://localhost:8888/w/extensions/MobileFrontend/cached.html

I was making a case of enabling this on Hebrew wikipedia instead of Russian since it's a group 1 wiki, but then discovered a RTL, so roll out is blocked again.

Hebrew Wikipedia is a fine candidate, too. Sounds like T142016: Beta: Cannot access languages on main page and T142044: PointerOverlay misbehaves on RTL need to be in place in conjunction with or ahead of this one-wiki change. Feel free to update the Description with Hebrew Wikipedia, or do both Hebrew and Russian.

Change 302999 had a related patch set uploaded (by Jdlrobson):
Promote language switcher to top of page in Russian Wikipedia

https://gerrit.wikimedia.org/r/302999

I'm going to SWAT the fix to PointerOverlay today and then this
on Monday

Change 302999 merged by jenkins-bot:
Promote language switcher to top of page in Russian Wikipedia

https://gerrit.wikimedia.org/r/302999

Mentioned in SAL [2016-08-08T23:19:28Z] <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Promote language switcher to top of page on ruwiki (T138961) (duration: 00m 59s)

Some simple non-SQL join'd funnel analysis now that the change is coming into fuller force on ru.m.wikipedia.org is below. The following was with less than an hour remaining on 20160809. It's not worth drawing any conclusions right now other than that the eventing seems to look fairly healthy.

mysql:research@s6-analytics-slave.eqiad.wmnet [log]> select left(timestamp,8) ts, event_languageButtonVersion, event_event, count(*)
    -> from MobileWebLanguageSwitcher_15302503
    -> where timestamp > '201608'
    -> and webHost = 'ru.m.wikipedia.org'
    -> group by ts, event_event, event_languageButtonVersion
    -> order by ts, event_event, event_languageButtonVersion;
+----------+-----------------------------+--------------------------+----------+
| ts       | event_languageButtonVersion | event_event              | count(*) |
+----------+-----------------------------+--------------------------+----------+
| 20160801 | NULL                        | exitModal                |       25 |
| 20160801 | NULL                        | languageButtonImpression |     2713 |
| 20160801 | bottom-of-article           | languageButtonTap        |       30 |
| 20160801 | top-of-article              | languageButtonTap        |        1 |
| 20160801 | NULL                        | languageListLoaded       |       43 |
| 20160801 | NULL                        | pageLoaded               |     7005 |
| 20160801 | NULL                        | startLanguageSearch      |        1 |
| 20160802 | NULL                        | exitModal                |       28 |
| 20160802 | NULL                        | languageButtonImpression |     2689 |
| 20160802 | bottom-of-article           | languageButtonTap        |       42 |
| 20160802 | top-of-article              | languageButtonTap        |        1 |
| 20160802 | NULL                        | languageListLoaded       |       45 |
| 20160802 | NULL                        | pageLoaded               |     6977 |
| 20160802 | NULL                        | startLanguageSearch      |        1 |
| 20160803 | NULL                        | exitModal                |       16 |
| 20160803 | NULL                        | languageButtonImpression |     2736 |
| 20160803 | bottom-of-article           | languageButtonTap        |       23 |
| 20160803 | top-of-article              | languageButtonTap        |        1 |
| 20160803 | NULL                        | languageListLoaded       |       26 |
| 20160803 | NULL                        | pageLoaded               |     7011 |
| 20160804 | NULL                        | exitModal                |       27 |
| 20160804 | NULL                        | languageButtonImpression |     2739 |
| 20160804 | bottom-of-article           | languageButtonTap        |       41 |
| 20160804 | NULL                        | languageListLoaded       |       41 |
| 20160804 | NULL                        | pageLoaded               |     7019 |
| 20160805 | NULL                        | exitModal                |       20 |
| 20160805 | NULL                        | languageButtonImpression |     2653 |
| 20160805 | bottom-of-article           | languageButtonTap        |       26 |
| 20160805 | top-of-article              | languageButtonTap        |        6 |
| 20160805 | NULL                        | languageListLoaded       |       34 |
| 20160805 | NULL                        | pageLoaded               |     6874 |
| 20160806 | NULL                        | exitModal                |       22 |
| 20160806 | NULL                        | languageButtonImpression |     2924 |
| 20160806 | bottom-of-article           | languageButtonTap        |       25 |
| 20160806 | top-of-article              | languageButtonTap        |        3 |
| 20160806 | NULL                        | languageListLoaded       |       36 |
| 20160806 | NULL                        | pageLoaded               |     7635 |
| 20160807 | NULL                        | exitModal                |       31 |
| 20160807 | NULL                        | languageButtonImpression |     3227 |
| 20160807 | bottom-of-article           | languageButtonTap        |       32 |
| 20160807 | top-of-article              | languageButtonTap        |        1 |
| 20160807 | NULL                        | languageListLoaded       |       39 |
| 20160807 | NULL                        | pageLoaded               |     8156 |
| 20160807 | NULL                        | startLanguageSearch      |        1 |
| 20160808 | NULL                        | exitModal                |       23 |
| 20160808 | NULL                        | languageButtonImpression |     2826 |
| 20160808 | bottom-of-article           | languageButtonTap        |       24 |
| 20160808 | top-of-article              | languageButtonTap        |        1 |
| 20160808 | NULL                        | languageListLoaded       |       34 |
| 20160808 | NULL                        | pageLoaded               |     7272 |
| 20160808 | NULL                        | startLanguageSearch      |        3 |
| 20160809 | NULL                        | exitModal                |       17 |
| 20160809 | NULL                        | languageButtonImpression |     4841 |
| 20160809 | bottom-of-article           | languageButtonTap        |       11 |
| 20160809 | top-of-article              | languageButtonTap        |       11 |
| 20160809 | NULL                        | languageListLoaded       |       31 |
| 20160809 | NULL                        | pageLoaded               |     7564 |
+----------+-----------------------------+--------------------------+----------+

Did impressions of traffic go up?
Do we need to think about adjusting sampling rate? (Looks like it's getting clicked a lot more now?)

The ratio of impressions to page loads has gone up, as expected. It looks like the button tap to page load ratio is in the same ballpark right now, which is acceptable. Even if the tap ratio stays about the same, we've made it a lot easier for users to get to it. Granted, this is at the expense of future options for buttons in the page action bar, but it's okay.

I believe we can keep the sampling rate as is. It's true "unnecessary" button impression events will be fired, but for the time being this is the least complicated approach. Eventually we'll want to remove the eventing for these funnels generally anyway.

The ratio of impressions to page loads has gone up, as expected. It looks like the button tap to page load ratio is in the same ballpark right now, which is acceptable.

This is what I saw when I ran an informal query on beta--there didn't seem to be an impact on language clicks, which is surprising since we moved something from the bottom to the top of the page. 3 hypotheses:

• Since we moved it to the top, we simplified the button from: "read in another language" to the much simpler: [symbol]. It might be that it will take some time for people to learn what [symbol] means.
• It also might be that there is 0 volume impact because the people who would switch were willing to crawl through the mud to do it....and we simply made it a lot easier for them.
• The last hypothesis I have is negative, which is that some large % of the time people realize they want to switch only when they reach the end of the article and are unsatisfied. If this is the case, they now have to scroll up and we have made their lives harder. I do not think this is what is happening. The lack of volume change (and our instinct) does not support this. In hindsight, more clarity would have come if we added this question to the design research we did or talked more with target users.