Page MenuHomePhabricator

Investigate referrer class change on Chrome Mobile from September 13, 2018
Closed, ResolvedPublic

Description

From September 13 on, pageviews with "unknown" referrer from Chrome Mobile dropped sharply, to almost a third of their previous level.[1] (Similar on desktop Chrome, although there the absolute number was much smaller to begin with.[1])
It looks like this coincided with an increase in "external (search engine)" referrers on Chrome Mobile.[2]

This change does not seem to have come from the rollout of a new Chrome Mobile version.[3]

There are no corresponding entries in https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Webrequest#Changes_and_known_problems_since_2015-03-04 or related changelogs.

[1]:

Unknown referrers per browser, January-November 2018.png (859×1 px, 122 KB)

Source: Turnilo

[2]:

external (search engine) referrers from Chrome Mobile, January-November 2018.png (855×1 px, 73 KB)

Source:
Turnilo

[3]:

Unknown referrers from Chrome Mobile, by version (64-72), January-November 2018.png (847×1 px, 142 KB)

Source: Turnilo

Event Timeline

It does coincide with an upgrade, right? All those chrome mobile views are coming from Android and Android 8 overtakes all other versions about when. 8.0 was released (August 21st is public release, until then it was in beta) https://en.wikipedia.org/wiki/Android_Oreo

You can see how Android8 deploy progresses and thus increases the presence of more proper referrers:

Screen Shot 2018-12-06 at 10.14.48 PM.png (1×2 px, 393 KB)

https://bit.ly/2rri1eA

Screen Shot 2018-12-06 at 10.24.26 PM.png (1×2 px, 399 KB)

https://bit.ly/2SvSZ9R

While android 8 plays a factor you can also see below Android 8 referrers switching from unknown to "search engine" at about Sep 9th so there is more to this:

Screen Shot 2018-12-07 at 1.06.55 PM.png (1×2 px, 427 KB)

If you split by country you call tell this sharp decline is not so sharp in India, which points to an upgrade of some sort (the google app?) as upgrades are slower in India than other countries

Screen Shot 2018-12-07 at 1.08.28 PM.png (1×2 px, 386 KB)

https://bit.ly/2PswS22

While change seems sharp on the 12 months plot it actually happens in the course of couple weeks from September 9th to September 22nd. The fact that is only present on Android Chrome mobile and that (see above) the change is slower in some countries in which we know upgrades are slower points to a security patch or upgrade on Android. The android security patch released on sep 9th: https://forums.androidcentral.com/google-pixel-2-pixel-2-xl/907638-september-2018-update-6.html matches pretty well in terms of dates so I bet that is the root cause of the shift.

See patches:

https://source.android.com/security/bulletin/2018-10-01.html

fdans triaged this task as Medium priority.
fdans moved this task from Incoming to Data Quality on the Analytics board.