Ooh, it's even worse. The conversion they do somehow makes the entire page link to the amazon form, moving the link outside the CentralNotice div. I have emailed googleweblight@google.com for guidance, and cc'd fr-tech.

I moved this back into our triage column. We'll look at it today.

@AndyRussG I tried adding the following js, but it didn't work. The banner still gets displayed.

if ( document.location.hostname === 'googleweblight.com' ) {
    mw.centralNotice.bannerData.hideResult = true;
}

if (!mw.centralNotice.bannerData.hideResult) {
    fundraisingBanner.showLargeBanner();
}

Another reason not to show them a banner: they're getting geolocated to the US, when it looks from x-forwarded-for headers like a large percentage of them are actually in other countries.

I think we're not running any of our JS in the browsers of end-users when they see pages munged like this. So, we have to block somehow when Google is scraping our content. We could do this server-side or client-side (i.e., in whatever browser or spider Google is using)... It's just that we need to know how to detect it, and if it's client-side, ensure the detection doesn't slow things down for everyone else.

From the doc:

If you do not want your pages to be transcoded, set the HTTP header "Cache-Control: no-transform" in your page response. If Googlebot sees this header, your page will not be transcoded.

Dunno if we'd want to block transformations for the entire site... We could easily add this header when we serve banners from Special:BannerLoader, though. I don't know if that would make Google not load the banner, or somehow munge it less... Or maybe it wouldn't make any difference at all.

@Pcoombe Could you try hiding with a condition like this, maybe? Thanks!!!!

If ( navigator.userAgent.indexOf( 'googlweblight' ) !== -1 ) {
   ...hide...
} else {
   ...don't hide...
}

Also, I wouldn't be surprised if it doesn't take some time for a change to appear... I don't know how often Google scrapes content for this (if that's what they do). (I tried viewing a test site on a VPS through googleweblight, but it showed a blank page, and I didn't see any requests from google in the server log.)

In beacon/impression server logs for December 7, I found 2,652,987 requests (quite a lot!!! :( ) with the following user agent string:

Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 5 Build/JOP40D) AppleWebKit/535.19 (KHTML, like Gecko; googleweblight) Chrome/38.0.1025.166 Mobile Safari/535.19

We should probable take a more hardened route and add a Varnish rule to block this UA for Special:BannerLoader... What do u think?

Okay, hiding based on userAgent appears to be working in a test banner on aa.wikibooks.org. I'll roll this out to our other banners and hopefully we'll start to see a drop in impressions/Amazon clicks.

@Pcoombe Fantastic, thanks!!! We are also working on a server-side Varnish-based option... But, I imagine the in-banner option will roll out faster, so I think going ahead with that is great :)

In T152602#2869744, @AndyRussG wrote:

I tried viewing a test site on a VPS through googleweblight, but it showed a blank page, and I didn't see any requests from google in the server log

Correction: I was using the wrong URL. Now, I am seeing a request in the server log for every view through the service...

Hmmm... I added the same code

if ( navigator.userAgent.indexOf( 'googleweblight' ) !== -1 ) {
   mw.centralNotice.bannerData.hideResult = true;
}

to the mobile banners currently up in Big English about an hour ago. While this worked on aa.wikibooks.org, it doesn't seem to be making a difference yet at https://googleweblight.com/?lite_url=https://en.m.wikipedia.org. Faking the googleweblight user agent for my browser at https://en.m.wikipedia.org does hide the banner, so it doesn't look like an issue with caching on our end.

Confirmed that the userAgent seen by JS is the same as that sent in the header: https://googleweblight.com/?lite_url=http://ejegg.com/mediawiki/blight.html

UserAgent is: Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 5 Build/JOP40D) AppleWebKit/535.19 (KHTML, like Gecko; googleweblight) Chrome/38.0.1025.166 Mobile Safari/535.19

Change 327043 merged by jenkins-bot:
Add googleweblight to JS blacklist

https://gerrit.wikimedia.org/r/327043

Good news, I'm no longer seeing a banner at https://googleweblight.com/?lite_url=https://en.m.wikipedia.org, and the number of entries in contribution_tracking with referrer='googleweblight.com/' has dropped hugely today!

It was pointed out in IRC that certain test links like https://googleweblight.com/?lite_url=https://en.wikipedia.org?country=CA still show a banner. For some reason it seems adding that causes weblight to load the desktop site and banners, where I hadn't put in the useragent check. Will add to those now.

Thanks! We're rolling out the patch to treat weblight as a non-JS browser in an hour and 15 minutes, so we shouldn't need any special code in banners after that.

Change 327250 had a related patch set uploaded (by Ejegg):
Add googleweblight to JS blacklist

https://gerrit.wikimedia.org/r/327250

Change 327250 abandoned by Ejegg:
Add googleweblight to JS blacklist

https://gerrit.wikimedia.org/r/327250

Change 327251 had a related patch set uploaded (by Ejegg):
Add googleweblight to JS blacklist

https://gerrit.wikimedia.org/r/327251

Change 327252 had a related patch set uploaded (by Ejegg):
Add googleweblight to JS blacklist

https://gerrit.wikimedia.org/r/327252

Change 327251 merged by jenkins-bot:
Add googleweblight to JS blacklist

https://gerrit.wikimedia.org/r/327251

Change 327252 merged by jenkins-bot:
Add googleweblight to JS blacklist

https://gerrit.wikimedia.org/r/327252

Mentioned in SAL (#wikimedia-operations) [2016-12-14T19:32:45Z] <thcipriani@tin> Synchronized php-1.29.0-wmf.6/resources/src/startup.js: SWAT: [[gerrit:327252|Add googleweblight to JS blacklist]] T152602 (duration: 00m 41s)

Mentioned in SAL (#wikimedia-operations) [2016-12-14T19:34:20Z] <thcipriani@tin> Synchronized php-1.29.0-wmf.5/resources/src/startup.js: SWAT: [[gerrit:327251|Add googleweblight to JS blacklist]] T152602 (duration: 00m 39s)

Just checking the impression data in Hive... It looks like the in-banner JS fix didn't work, but the core patch did. It's still a little early to confirm the latter. I'll check again in the morning when we have more data.

Regarding the in-banner JS, I would have expected to see impressions with statusCode 5 if the banner had been successfully hidden by that method. @Pcoombe, could you please point me to some versions of banners that included that code? (I think it's not in the banners that are up now? Or maybe I missed something?)

Thanks!!!

In T152602#2875320, @AndyRussG wrote:

It looks like the in-banner JS fix didn't work, but the core patch did.

Another possibility is that it did work, but it's not setting status for beacon/impression as expected when it hides the banner.

Strangely, I'm seeing a number of waitimps statuses (2.3) mostly below 400 imp/hr, but with two spikes, up to 10600 imp/hr and 8800 imp/hr, at 15 hrs and 19 hrs today, respectively. So, somehow these bots must be storing data between "views". Bizarre! Maybe they store cookies or localstorage values "on behalf" of end users served?

Hmmm, the numbers are down substantially (about 20% of where we were yesterday) but there are still a lot of impressions coming in via this proxy...

BTW, I'm not seeing any banners on pages I visit through the proxy anymore...

Here are some of the points discussed in IRC about this issue:

• A server-side strategy was laid out: a Varnish rule could be added for Special:BannerLoader to return hard-coded string with a call to mw.centralNotice.handleBannerLoaderError() instead of any banner content for this client.
• Blocking ChoiceData might have been a more elegant option, in terms of CentralNotice workflow, however that's normally sent to the client all bundled up with other ResourceLoader stuff, so it'd require splitting the Varnish cache on this UA for all RL content... not really feasible.
• Since none of our JS is running in the end user's browser, it seemed more reasonable to just turn off Javascript for this client.
• There's an easy way to do this, and we're already doing it for other similar clients (like Opera Mini).
• Since the WMF is committed to providing a good reading experience on non-JS clients, and in any case, you can't do much besides read articles via googleweblight, this should be fine.
• There was some discussion among developers on other teams about whether there might not be some advantages to allowing JS to run. One possible issue is Common.js, a wiki page with JS that often modifies the appearance of the wiki.
• Also, the proxy is a black box from our perspective, so it's really hard to know what it'll do and what will work.
• Still, we went ahead with blocking JS on googleweblight.
• If this client-side solution doesn't work, or doesn't work well enough, we can go ahead with the Varnish method, too.
• An issue with the proposed Varnish method is that it hardcodes a CN-specific JS call in a completely separate codebase. However, the alternative server-side approach, splitting the Varnish cache on this UA just for Special:BannerLaoder, would be more complex.

The JS blacklist is working!

Till this morning, requests for https://googleweblight.com/?lite_url=https://en.wikipedia.org/wiki/Barack_Obama were coming back with all the images below the fold deleted (since our mobile site only loads them when the user scrolls down), and with the content div's class changed to "client-js" (which our JS does in browsers we deem worthy).

Now, requests to that URL come back with all images included, and with the content div's class "client-nojs", indicating that we're no longer running all the javascript on the proxy.

@Pcoombe, we should be able to take all the special-purpose code out of the banners now.

Thanks for all the help everyone! The random contribution_tracking entries from this referrer are down to practically zero.

@AndyRussG Here's an example banner with the code: https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/Edit/B1516_121315_en6C_mob_p1_lg_dsn_2016
I didn't set a hideReason since afaik those need to be added server-side too, and didn't know that any of the existing ones was suitable. Would that be an issue? Not that it matters much now anyway.

In T152602#2878773, @Pcoombe wrote:

@AndyRussG Here's an example banner with the code: https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/Edit/B1516_121315_en6C_mob_p1_lg_dsn_2016
I didn't set a hideReason since afaik those need to be added server-side too, and didn't know that any of the existing ones was suitable. Would that be an issue? Not that it matters much now anyway.

Thanks much, @Pcoombe! :) That code looks fine. :) It's hard to know what the proxy was doing and why we didn't results with this approach. As has others have said, it's a "black box" for us. Also, it seems they have their own internal caching layers.

(BTW, setting a reason is not required. However, it's also OK to use a new reason string for such cases, I think. The string will be sent back with beacon/impression, and the statusCode should be 5.0, that is, BANNER_LOADED_BUT_HIDDEN.other.)

Looking at beacon/impression data from December 18 (a few days after the fix was deployed, campaign still up) shows exactly 0 impressions from this proxy! :) So, I'm closing the task... Thanks much, all!! :)