Page MenuHomePhabricator

Enable RESTBase for mobile sites, or support zero headers in text varnishes
Closed, DuplicatePublic

Description

The new app service will need a way to detect requests from zero rated networks. This functionality is currently specific to the mobile sites. To provide the same functionality with RESTBase, there seem to be two main options:

  1. add the /api/rest_v1/ entry points in the mobile varnish configs as well, so that they get the zero detection from there, or
  2. add the zero detection to general text varnishes

Event Timeline

GWicke raised the priority of this task from to Needs Triage.
GWicke updated the task description. (Show Details)
GWicke added projects: RESTBase, Mobile-Apps.
GWicke added subscribers: GWicke, bearND, BBlack.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 15 2015, 5:49 PM
GWicke triaged this task as Medium priority.Jun 15 2015, 5:49 PM
GWicke updated the task description. (Show Details)
GWicke set Security to None.
GWicke added a project: Traffic.
Yurik added a subscriber: Yurik.Jun 15 2015, 7:39 PM

#2 has been requested in T89177 a long time ago, and would be very good for analytics as well. There will be no extra headers sent to the non-zero partners if all the traffic goes through the detection.

GWicke added a comment.EditedJun 15 2015, 9:04 PM

@Yurik, interesting. Thank you for the pointer!

@BBlack: I'm very curious about your view on this issue, once you get to catch some breath after the HTTPS roll-out.

Kelson added a subscriber: Kelson.

Can you make sure the Beta-Cluster-Infrastructure is updated as well? Might be worth a sub task.

Special:NovaProxy points restbase-beta.wmflabs.org to http://deployment-restbase01.eqiad.wmflabs:7231

And Special:NovaAddress has a bunch of tricky DNS wildcard mapping to the Varnish caches:

DNSCache
*.m.wikipedia.beta.wmflabs.orgdeployment-cache-mobile03
*.zero.wikipedia.beta.wmflabs.orgdeployment-cache-mobile03
*.beta.wmflabs.orgdeployment-cache-text02

(lot of entries left)

phuedx added a subscriber: phuedx.Jun 16 2015, 9:15 AM

Can you make sure the Beta-Cluster-Infrastructure is updated as well? Might be worth a sub task.
Special:NovaProxy points restbase-beta.wmflabs.org to http://deployment-restbase01.eqiad.wmflabs:7231

Yup, that's for external access to RESTBase.

And Special:NovaAddress has a bunch of tricky DNS wildcard mapping to the Varnish caches:

DNSCache
*.m.wikipedia.beta.wmflabs.orgdeployment-cache-mobile03
*.zero.wikipedia.beta.wmflabs.orgdeployment-cache-mobile03
*.beta.wmflabs.orgdeployment-cache-text02

(lot of entries left)

These pertain to MobileWeb, not MobileApps from what I can tell.

GWicke renamed this task from Enable RESTBase for mobile sites for zero headers to Enable RESTBase for mobile sites, or support zero headers in text varnishes.Jun 16 2015, 8:44 PM
Restricted Application added a subscriber: Matanya. · View Herald TranscriptJun 16 2015, 8:44 PM

These pertain to MobileWeb, not MobileApps from what I can tell.

My understanding is that they are relevant to both apps *and* mobile web.

These pertain to MobileWeb, not MobileApps from what I can tell.

My understanding is that they are relevant to both apps *and* mobile web.

Technically, this for both mobile web and apps. It probably doesn't make a difference for ops but initially it would be only apps using RESTBase. Mobile web might come later.

@BBlack, I'm curious about your thoughts on enabling the zero detection in text varnishes. Do you think that's a good direction to go in?

Yeah, I think it is, and I think we're going there for reasons orthogonal to RB anyways (see also T89177). Basically, there's no good reason not to be putting the X-CS / X-F-B / Via headers on all our inbound traffic, and also copying relevant bits over to X-Analytics as well. There's all sorts of potential uses for that information that are valid. There's still a few steps to go before we're actually ready to turn it on, though, which is now slowly happening as a component of T109286, mostly.

@BBlack: Thanks, that's great news. I'll mark this task as a dupe of T89177, then.

GWicke closed this task as a duplicate of Restricted Task.Aug 21 2015, 4:03 PM
BBlack moved this task from Triage to Done on the Traffic board.Aug 27 2015, 2:41 AM
bearND awarded a token.Feb 5 2016, 6:36 PM

I would like to see this implemented. T89177 is not public though.

BBlack added a comment.Feb 5 2016, 6:43 PM

@bearND - you'd like to see what implemented, that isn't already?

bearND added a comment.Feb 5 2016, 7:29 PM

@BBlack I'd like to see the W0 header (X-CS) added to API responses for regular domains, in addition to the m-dot domains.
So, when I check headers for the following URIs I should be getting them for all requests. (I have a W0 test IP at home).

MW API

curl -sI https://en.m.wikipedia.org/w/api.php?action=mobileview&page=Cat | grep X-CS

:: m-dot gets X-CS header

curl -sI https://en.wikipedia.org/w/api.php?action=mobileview&page=Cat | grep X-CS

:: non-m-dot does not get X-CS header

RB

curl -sI https://en.wikipedia.org/api/rest_v1/page/html/Cat | grep X-CS

:: RB does not get X-CS header

BBlack added a comment.Feb 5 2016, 7:39 PM

I'll make a separate task about that. In theory it's trivial to enable, but it probably needs some coordination with various parties to make sure it doesn't cause unintended side-effects...