We have a huge amount of apache rewrites, it'd be really nice if we had a way to test them so that we know for an input domain we get an output domain
I think there is some simplification that can be done by ordering the rewrites, and then needing less conditionals on some of the later rewrites etc
Version: wmf-deployment
Severity: enhancement
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=43266
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| httpbb: Add a test verifying query string behavior in the TLS redirect. | operations/puppet | production | +8 -0 |
Idea worth to discuss imo: setting LogLevel rewrite:trace8 in the apache logs (only for testing) is really useful:
elukey@deployment-mediawiki06:~$ curl http://localhost --header "Host: wikipedia.beta.wmflabs.org"
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://www.wikipedia.beta.wmflabs.org/">here</a>.</p>
</body></html>
elukey@deployment-mediawiki06:/var/log$ sudo tail -f apache2.log | grep "\[rewrite:" | cut -d " " -f 16-
applying pattern '^(.*)$' to uri '/wiki/Main_Page'
RewriteCond: input='en.wikipedia.beta.wmflabs.org' pattern='=commons.wikipedia.beta.wmflabs.org' => not-matched
pass through /wiki/Main_Page
init rewrite engine with requested uri /
applying pattern '.' to uri '/'
setting env variable 'RW_PROTO' to ''
applying pattern '.' to uri '/'
RewriteCond: input='' pattern='!=https' => matched
setting env variable 'RW_PROTO' to 'http'
applying pattern '^/([a-z]{2}|meta)/(.*)$' to uri '/'
applying pattern '/apple-app-site-association' to uri '/'
applying pattern '^(.*)$' to uri '/'
rewrite '/' -> 'http://www.wikipedia.beta.wmflabs.org/'
explicitly forcing redirect with http://www.wikipedia.beta.wmflabs.org/
escaping http://www.wikipedia.beta.wmflabs.org/ for redirectelukey@deployment-mediawiki06:/var/log$ curl http://localhost --header "Host: en.wikipedia.beta.wmflabs.org" -i HTTP/1.1 301 Moved Permanently Date: Wed, 16 Nov 2016 21:00:21 GMT Server: deployment-mediawiki06.deployment-prep.eqiad.wmflabs X-Powered-By: HHVM/3.12.7 X-Content-Type-Options: nosniff Cache-control: s-maxage=1200, must-revalidate, max-age=0 P3P: CP="This is not a P3P policy! See https://en.wikipedia.beta.wmflabs.org/wiki/Special:CentralAutoLogin/P3P for more info." Vary: Accept-Encoding,X-Forwarded-Proto,Cookie,Authorization Location: https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page Last-Modified: Wed, 16 Nov 2016 21:00:21 GMT Backend-Timing: D=66998 t=1479330021366903 Content-Length: 0 Content-Type: text/html; charset=utf-8 elukey@deployment-mediawiki06:/var/log$ sudo tail -f apache2.log | grep "\[rewrite:" | cut -d " " -f 16- applying pattern '^/robots\\.txt$' to uri '/wiki/Main_Page' applying pattern '^/$' to uri '/wiki/Main_Page' applying pattern '^/favicon\\.ico$' to uri '/wiki/Main_Page' applying pattern '^/apple-touch-icon\\.png$' to uri '/wiki/Main_Page' applying pattern '^/w/skins/.*$' to uri '/wiki/Main_Page' applying pattern '^/w/resources/.*$' to uri '/wiki/Main_Page' applying pattern '^/w/extensions/.*$' to uri '/wiki/Main_Page' applying pattern '^(.*)$' to uri '/wiki/Main_Page' RewriteCond: input='en.wikipedia.beta.wmflabs.org' pattern='=commons.wikipedia.beta.wmflabs.org' => not-matched pass through /wiki/Main_Page init rewrite engine with requested uri / applying pattern '^/$' to uri '/' rewrite '/' -> 'fcgi://127.0.0.1:9000/srv/mediawiki/docroot/wikipedia.org/w/index.php' forcing proxy-throughput with fcgi://127.0.0.1:9000/srv/mediawiki/docroot/wikipedia.org/w/index.php
It might be hacky but we could use mod_rewrite's error log output to unit test a specific set of HTTP requests that we expect Apache to handle in a specific way (not only returning the expected result but also with the expected steps).
operations/apache-config.git had a test suite written by Tim. Based on curl it was exercising the apache conf quite nicely. You can find it in origin/HEAD^ in /test/ . Maybe we can try hooking it in Jenkins.
I also tried to do get a script doing an Apache config check against both puppet.git and mediawiki-config.git T72068 ( lame dirty code for Jenkins is https://gerrit.wikimedia.org/r/#/c/166033/2/jjb/operations-apache-config.yaml ). Did not go well though :-(
Change 556989 had a related patch set uploaded (by RLazarus; owner: RLazarus):
[operations/puppet@production] httpbb: Add a test verifying query string behavior in the TLS redirect.
Change 556989 merged by Giuseppe Lavagetto:
[operations/puppet@production] httpbb: Add a test verifying query string behavior in the TLS redirect.