Page MenuHomePhabricator

Fix broken referer categorization for visits from Safari browsers
Closed, ResolvedPublic


Due to a change we made to our meta referrer tag in February of 2016 (T87276) and a bug in safari, almost all of the internally referred traffic on Safari is miscategorized. Since Safari is our second biggest browser family at ~20% of traffic, this impacts our non-internal referral count which is an essential measure of product health as it is a rough proxy for visits. The ratio is also an important proxy for session depth. This has been broken for almost a year and we should fix it or put more pressure on the webkit folks to fix it on their end:

I do not know how to fix it, but the original change is heavily discussed here T87276 and the implications for our metrics are outlined here: T148780

Event Timeline

Ottomata triaged this task as Medium priority.Mar 6 2017, 7:29 PM
Ottomata subscribed.

I don't fully understand this issue, but I'm not sure how it could be fixed on our side. If there is a way, its likely to be very complicated. I'm doing some ops triaging this week, and I'm not sure what to do with this ticket.

@Ottomata Thanks for voicing your uncertainty. I am also uncertain/confused about the cause (T87276), or the solution as it is a bit out of my technical depth, but it seems that there is a typo in webkit and I don't understand why we can't adjust for that typo.

On our end we have no preference how to fix issue (adding typo back would break other things, but I do not dispute that it might be fine), traffic & reading (and likely performance) just need to agree how to move forward, analytics will work automagically, so there is nothing on the analytics side to fix, the issue is on the mediawiki headers which are (i guess) collectivelly owned by all teams. I think the shortest path to action would be for a mediawiki reading developer to drive issue.

I was going to summarize but @mforns already did it in a prior ticket. For what is causing this issue see:

The workarrounds to fix it could be several:

  1. changing to the typo'd version of meta tag, that be changing origin-when-cross-origin,to origin-when-crossorigin on this file: This would mean that Safari sends the right referrer but other browsers might not recognize the typo-ed version (browsers moved to version w/o typo a while back with the notable exception of safari) so we might need both versions of the meta tag to be available.
  1. do not use meta tags and use headers in VCL to carry this information using the new Referrer policy:

Both 1 and 2 need browser testing to assess level of support, probably 1) is less work as of now if we can get away with typo-ed version, best choice however is to do 2) as it allows us to modify that header on VCL when we need to even if html is cached. Testing for either option would need to happen in all popular browsers:

As far as i'm aware, this is NOT due to a typo. Safari simply implements a very limited set of referrer policies: 'no-referrer', 'origin', 'no-referrer-when-downgrade', or 'unsafe-url'.

I also wonder about IE. As far as I know it also doesn't support this specific policy, but perhaps it falls back to a more 'leaky' referrer policy.

The Referrer-Policy HTTP Header is actually quite different from the meta header. I have created a mock request, but it doesn't seem to me that Safari does anything at all with it. I get the full referer in my request console when I click the link.

As far as i'm aware, this is NOT due to a typo. Safari simply implements a very limited set of referrer policies: 'no-referrer', 'origin', 'no-referrer-when-downgrade', or 'unsafe-url'.

I see, then solution 1) should work for Safari if we can render a different meta tag for safari, is that aceptable? cc @TheDJ

@JKatzWMF, I think if you really really really want this fixed, you'll need to find a MediaWiki dev to revisit, involve the folks who made this decision in the first place, and then figure out what to do and do it.

Does reading have a developer that could shepherd this?

@Ottomata @Nuria Just spoke with @Nuria and between the above comments and our conversation I think I have what we need to figure out next steps....which are probably conversations with other folks like Dario and the discovery team. I really appreciate the work that went into exploring and communicating this.

you'll need to find a MediaWiki dev to revisit

I *think* policy as is works, we will just need to special case safari seems like

+1 to @Nuria 's comment: I think the main concern here from @DarTar and me is that external websites need to have some sort of awareness where the dark traffic is coming from: so if it messes with external analytics tools for referrals to other sites, it is a less-than-good solution.

Safari Technology Preview v38, released 2 days ago, seems to be solving this. I at least am no longer getting the console warning that the policy is unsupported by the browser (have not verified yet, if the behaviour is now actually correct).

This might mean that the next major version of Safari on desktop and mobile, could start showing up in the referrer stats.

@TheDJ : noted, please ping us when those changes make it into safari's main version. The bulk of our views is on Safari 10 so if problem is fixed on next version we should see our referrer stats change

I have now tested this and the preview does indeed implement origin-when-cross-origin as directed by our meta referrer header.

FYI: Seems the change missed the cut-off for the Safari 11 release.

iOS 11 Safari 11 not yet fixed: Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1
macOS Safari 11 not yet fixed: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Safari/604.1.38

So possibly no earlier than the March 2018 for this change (Safari 11.1)

@Nuria so this should work now, can we confirm that from the stats ?

TheDJ claimed this task.

I think we can call this one resolved, per T180921#4226106

Internal_refferers_safari.png (1×2 px, 470 KB)

@TheDJ is there a way to know when the fix landed in the safari version users have (not when it was merged and such). I just want to .. ahem... triple check that bump we see around March 29th.

@Nuria Safari 11.1 was released with iOS 11.3 and macOS 11.3.4 (as well as macOS 10.12.6 and 10.11.6)

Both realeased to the general public on March 29th

If we change the scale on that graph im sure we can find public beta and developer traffic starting early february.

PS this make me wonder how webkit technology preview is counted in our user agent stats......

@TheDJ Thanks for monitoring this and letting us know! It's a huge relief.

So nice when things add up! I can check on the preview staff, let me know if you want that done