Page MenuHomePhabricator

RESTBase CORS redirect resolve should not hit frontend caches
Closed, ResolvedPublic

Description

RESTBase supports following on-wiki redirects with 302 responses and a relative link in a Location header. However, in case of CORS, browsers can respond in a weird way to redirects, effectively making redirect pages in CORS unreachable T149295

As a mitigation for this, if RESTBase detects a cross-origin request, it resolves the redirect internally, returning 200 response with the content of the redirect target. However, to fetch the redirect content, it uses public RESTBase URI, thus it's hitting the frontend caches, and forwards the response as-is, with all the cache-related headers, X-Cache, Age, etc.

According to @ema this is causing problems for ATS. So, in order to mitigate it, we need to change the code implemented in https://github.com/wikimedia/restbase/pull/702 to fetch the redirect target content directly from RESTBase, not from the caches and forward it.

Event Timeline

For the record I don't think this currently causes any specific functional issue. I've spotted a few RESTBase responses coming out of ats-be with Age higher than 86400 (24h), which is our TTL cap at the ats-be layer. It took a while to understand that this wasn't due to a issue in the TTL cap itself but rather to the fact that the origin was returning Age. In general we like to rely on the fact that origin server responses are coming from the origin directly and are not going through the CDN frontend layer to keep things at least somehow sane. :)

Thanks for helping with the investigation @Pchelolo and for opening this task.

Mentioned in SAL (#wikimedia-operations) [2020-08-17T15:22:10Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@ddcecce]: T257943 T260556 T253478 T254490 T259054 (duration: 02m 30s)

Mentioned in SAL (#wikimedia-operations) [2020-08-17T15:22:25Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@ddcecce]: T257943 T260556 T253478 T254490 T259054. take 2. feeds timed out

Mentioned in SAL (#wikimedia-operations) [2020-08-17T15:43:04Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@ddcecce]: T257943 T260556 T253478 T254490 T259054. take 2. feeds timed out (duration: 20m 40s)

Mentioned in SAL (#wikimedia-operations) [2020-08-17T15:43:12Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@ddcecce]: T257943 T260556 T253478 T254490 T259054. take 3. feeds timed out

Mentioned in SAL (#wikimedia-operations) [2020-08-17T15:44:39Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@ddcecce]: T257943 T260556 T253478 T254490 T259054. take 3. feeds timed out (duration: 01m 31s)

Pchelolo claimed this task.
Pchelolo moved this task from Later to Done on the Platform Team Workboards (Clinic Duty Team) board.
curl -i -H 'origin: test' http://restbase.discovery.wmnet:7231/fr.wikipedia.org/v1/page/summary/La%20montagne | grep age:
 
content-language: fr

Fixed.