MCS doesn't set ETag if 'accept-encoding' header is provided.
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Pchelolo
	Oct 18 2016, 9:17 PM

Description

This is crazy, but when I ask MCS for the content via curl I get back an etag header:

curl -i http://appservice.wmflabs.org/en.wikipedia.org/v1/page/mobile-sections/User%3APchelolo%2FAccess_Check_Tests

BUT when I query the same URL with preq I don't get an etag header back.

preq.get({
    uri: 'http://appservice.wmflabs.org/en.wikipedia.org/v1/page/mobile-sections/User%3APchelolo%2FAccess_Check_Tests',
}).then(function (res) {
    console.log(res.headers.etag);
});

The problem is that preq is setting an Accept-Encoding: gzip, deflate header, which curl doesn't. Apparently the response of the Mobile-Content-Service is dependent on that header which it shouldn't be.

In production that's not happening, so apparently it's been broken in beta labs recently. This is a blocker for MCS deployment, because RESTBase really needs these etags to be set since our access control is based on them for mobile content.

Related Objects

Mentioned In: T150886: MCS should quote the ETag
T148788: Request increased quota for services labs project
T148592: Switch to restrictions-based access control
Mentioned Here: T150886: MCS should quote the ETag
T149671: Set up MCS in BetaCluster
T148788: Request increased quota for services labs project
T148676: Novaproxy in labs strips out the ETag header if gzip is enabled

Event Timeline

• Pchelolo created this task.Oct 18 2016, 9:17 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 18 2016, 9:17 PM

• Pchelolo renamed this task from MCS ETag weirness to MCS doesn't set ETag if 'accept-encoding' header is provided..Oct 18 2016, 9:19 PM

• Pchelolo triaged this task as High priority.

• Pchelolo added subscribers: • bearND, • Mholloway.

And also since we are in the etag business, the etag format is "rev_id/uuid" (note the double quotes). MCS returns the etag without double quotes.

• Pchelolo mentioned this in T148592: Switch to restrictions-based access control.Oct 18 2016, 9:28 PM

MCS has the latest master deployed, which means that both prod and labs MCS are at the exact same state. The difference here is that in labs, RB does not sit in between them, nor does Varnish, only nginx. Might it be that? Perhaps it's mangling the response and stripping the header off?

@mobrovac Ye, your theory is correct, it should be nginx, because directly querying the service from the node it's running on gives me back the etag. It's only left to find the labs nginx config..

In T148590#2727275, @mobrovac wrote:

MCS has the latest master deployed, which means that both prod and labs MCS are at the exact same state. The difference here is that in labs, RB does not sit in between them, nor does Varnish, only nginx. Might it be that? Perhaps it's mangling the response and stripping the header off?

This is the likely explanation. Using preq directly on the Labs node yields an ETag.

• AlexMonk-WMF subscribed.Oct 19 2016, 6:51 PM

So, before T148676 is resolved, the only way to workaround this is to assign a public IP to the service and bypass the proxy. However, the labs MCS instance was created in services project and labs restbase instance is in deployment-prep project, we wouldn't be able to use the public IP in labs RESTBase, so the same bug would be there.

So, to workaround the issue completely and correctly, we need:

Move MCS labs to deployment-prep project
Point labs RESTBase to the local IP of MCS
Assign a public IP to the MCS service and point local RESTBase testing to it.

@bearND @Mholloway Any insight why was mobileapps located at the services project and not in deployment-prep?

I agree that MCS should be in BetaCluster as well. Originally, the MCS team wanted to be able to test the service directly with the app, so it made sense to put it in the services project. That instance auto-updates on each commit, so as to allow them to live test MCS. I think we should keep it that way.

The reason why we have not (yet) set it up in Beta is because, until recently, MCS didn't work for beta domains. Since now it does, we should add it to deployment-sca0[12] (where most of the other SCB services live in Beta). That means that the MCS team will need to be responsible for updating it every time they do a deploy.

As for the public IP, I think we should assign it to appservice.services.eqiad.wmflabs nevertheless, because that is the freshest version of MCS, which ensures that we test RB against the newest one every time.

Ok, created a floating IP request T148788.

Would we deploy by updating the deploy repo and using scap3, like we do for production?

In T148590#2742933, @bearND wrote:

Would we deploy by updating the deploy repo and using scap3, like we do for production?

You mean for BetaCluster? Yes, the procedure is the same, except it should happen from deployment-tin.deployment-prep.wmflabs.

Yes, I meant Beta cluster. Thanks. Updated https://www.mediawiki.org/w/index.php?title=Wikimedia_Apps/Team/RESTBase_services_for_apps&section=19#Beta_cluster

Note that before you can actually deploy in Beta, we need to set it up - T149671: Set up MCS in BetaCluster

• bearND added a subtask: T149671: Set up MCS in BetaCluster.Nov 8 2016, 6:15 AM

Does MCS need to do anything here? Is this dependent on MCS running in beta cluster?

In T148590#2778713, @bearND wrote:

Does MCS need to do anything here?

Yup, this:

In T148590#2727138, @Pchelolo wrote:

And also since we are in the etag business, the etag format is "rev_id/uuid" (note the double quotes). MCS returns the etag without double quotes.

Is this dependent on MCS running in beta cluster?

No, but we do need to set it up there. Once the quotes are added to the ETag, we can resolve this.

Resolving since the remaining piece is tracked by T150886 now.

MCS doesn't set ETag if 'accept-encoding' header is provided.Closed, ResolvedPublicActions

Description

Related Objects

Event Timeline

MCS doesn't set ETag if 'accept-encoding' header is provided.
Closed, ResolvedPublic
Actions