Page MenuHomePhabricator

Rewrite http://download.wikimedia.org/mediawiki/ -> https://releases.wikimedia.org/mediawiki in less than 3 redirects
Open, LowestPublic

Description

If you download a tarball from the current urls in the release announcements, you end up with

I know there's different layers to this, but, this shouldn't need 3 redirects, 1 should suffice in the best case, 2 at worst.

And shouldn't the last one be 301 Moved Permanently too?

$ wget http://download.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz

--2015-11-25 23:27:22--  http://download.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz
Resolving download.wikimedia.org (download.wikimedia.org)... 2620:0:862:ed1a::1, 91.198.174.192
Connecting to download.wikimedia.org (download.wikimedia.org)|2620:0:862:ed1a::1|:80... connected.
HTTP request sent, awaiting response... 301 TLS Redirect
Location: https://download.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz [following]

--2015-11-25 23:27:22--  https://download.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz
Connecting to download.wikimedia.org (download.wikimedia.org)|2620:0:862:ed1a::1|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://dumps.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz [following]

--2015-11-25 23:27:22--  https://dumps.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz
Resolving dumps.wikimedia.org (dumps.wikimedia.org)... 2620:0:861:1:208:80:154:11, 208.80.154.11
Connecting to dumps.wikimedia.org (dumps.wikimedia.org)|2620:0:861:1:208:80:154:11|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://releases.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz [following]

--2015-11-25 23:27:23--  https://releases.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz
Resolving releases.wikimedia.org (releases.wikimedia.org)... 2620:0:862:ed1a::3:d, 91.198.174.217
Connecting to releases.wikimedia.org (releases.wikimedia.org)|2620:0:862:ed1a::3:d|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 25174530 (24M) [application/x-gzip]

Event Timeline

Reedy created this task.Nov 25 2015, 11:47 PM
Reedy raised the priority of this task from to Lowest.
Reedy updated the task description. (Show Details)
Reedy added a subscriber: Reedy.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 25 2015, 11:47 PM

I should note that I've fixed the script that makes some of the release announcement emails to use the resultant url in https://gerrit.wikimedia.org/r/255472 so future release will use the newer url.

Doesn't fix it for old announcement urls of course.

I'll endeavour to go through and fix up on mediawiki.org to reduce some of the incoming traffic too
https?://download.wikimedia.org/mediawiki
https?://dumps.wikimedia.org/mediawiki
http://releases.wikimedia.org/mediawiki

I guess this related to the trailing slash redirect being handled by Apache, which responses to the request from Varnish which is over HTTP. It's not aware of itself being exposed as HTTPS to the outside. It will go to HTTP, even if the incoming request was over HTTPS.

The same applies to most other directory slash handling on various misc services.

Though in modern browsers this is skipped as it will prevent the HTTP request and use HTTPS before touching the network thanks to HSTS.

  1. 301 Permanent Redirect to https://dumps.wikimedia.org/mediawiki (downloads -> dumps)
  2. 302 Temporarily Redirect to https://releases.wikimedia.org/mediawiki (dumps -> releases)
  3. 301 Permanent Redirect to http://releases.wikimedia.org/mediawiki/ (Apache adds slash, downgrade to HTTP)
  4. (307 Internal Redirect) to https://releases.wikimedia.org/mediawiki/ (Non-Authoritative-Reason:HSTS)
  • 200 OK
hashar updated the task description. (Show Details)Nov 26 2015, 9:18 AM
hashar set Security to None.
hashar added a subscriber: hashar.Nov 26 2015, 9:27 AM

Seems the wiki got updated as https://www.mediawiki.org/wiki/Download points to https://releases.wikimedia.org/..

Possibly we could short circuit trailing slash redirect and wildcard redirect straight to https://releases.wikimedia.org/mediawiki/$1

https://dumps.wikimedia.org/mediawiki
https://dumps.wikimedia.org/mediawiki/

I am not sure it is worth the time to fix them up though.

The related configuration seems to be Nginx [[ https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/modules/dumps/templates/nginx.dumps.conf.erb | modules/dumps/templates/nginx.dumps.conf.erb ]] which has:

rewrite ^/(other/)?mediawiki(|/.*)$                $scheme://releases.wikimedia.org/mediawiki$2 break;
greg moved this task from Backlog to INBOX on the Release-Engineering-Team board.Oct 17 2019, 5:24 PM