Page MenuHomePhabricator

Requests to MW 404 when on HTTPS
Closed, ResolvedPublic

Description

As part of the switch-over effort (cf. T199073: Perform a datacenter switchover (2018-19 Q1)), we are moving to encrypted connections to MediaWiki. Unfortunately, Proton's Chromium has a problem with our TLS CA, so it was instructed to ignore HTTPS protocol errors. Alas, this causes Chromium to receive 404s from MW:

_status: 404,
_url: 'https://api-rw.discovery.wmnet/w/index.php?title=Foo',
_fromDiskCache: false,
_fromServiceWorker: false,
_headers: 
 { status: '404',
   date: 'Tue, 28 Aug 2018 10:41:30 GMT',
   'content-type': 'text/html',
   'content-length': '930',
   server: 'mw1348.eqiad.wmnet',
   'last-modified': 'Tue, 17 Jul 2018 17:07:05 GMT',
   etag: '"3a2-57134f9c13cba"',
   'accept-ranges': 'bytes',
   'backend-timing': 'D=337 t=1535452890761446' },
_securityDetails: 
 SecurityDetails {
   _subjectName: 'api.svc.eqiad.wmnet',
   _issuer: 'Puppet CA: palladium.eqiad.wmnet',
   _validFrom: 1490711956,
   _validTo: 1648478356,
   _protocol: 'TLS 1.2' } }

I am not yet positive, but it seems that Chromium does not send the Host header, even though it's available in its request context:

Request {
  _requestId: '4A4EE83A1C289A5BAB65BDB7B81C2E13',
  _isNavigationRequest: true,
  _interceptionId: null,
  _allowInterception: false,
  _interceptionHandled: false,
  _response: [Circular],
  _failureText: null,
  _url: 'https://api-rw.discovery.wmnet/w/index.php?title=Foo',
  _resourceType: 'document',
  _method: 'GET',
  _postData: undefined,
  _headers: 
   { 'upgrade-insecure-requests': '1',
     'user-agent': 'Proton/WMF',
     'x-devtools-emulate-network-conditions-client-id': '3477EFE499310F132D0CFBE83115E8F3',
     host: 'en.wikipedia.org',
     'x-subdomain': '' } }

For the time being, we have Proton using the HTTP end point, but a solution needs to be found here.

Apart from finding a way to force Chromium to use our CA, two other possible solutions we might want to consider are:

  • Use MW in the local DC. Proton only ever asks for the HTML contents of a page, so it performs read-only requests. It is thus safe to use MW from the local DC, which would alleviate the need for using TLS.
  • Pass through nginx/Varnish when requesting the HTML. That is, instead of calling https://api-rw.discovery.wmnet, use https://{domain}/. This would ensure the proper certificates are downloaded, but causes indirections and makes the request chain more complex.

Event Timeline

mobrovac created this task.
Pchelolo subscribed.

Seems like it's been fixed, the only thing left to be done is to remove the hacky line from puppet.

Seems like it's been fixed, the only thing left to be done is to remove the hacky line from puppet.

https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/455565/ was that revert and has been merged back in 2018. So, should we resolve this?

mobrovac claimed this task.

Indeed all's good here.