port 80 traffic is still being handled by varnish-frontend. We must move this to ats-tls if we intend to use atskafka to provide data to the analytics cluster.
varnish currently returns a "301 TLS Redirect" for GET/HEAD requests and a "403 Insecure Request Forbidden" for any other HTTP method:
willikins:puppet vgutierrez$ curl http://en.wikipedia.org/wiki/Main_Page -v * Trying 91.198.174.192... * TCP_NODELAY set * Connected to en.wikipedia.org (91.198.174.192) port 80 (#0) > GET /wiki/Main_Page HTTP/1.1 > Host: en.wikipedia.org > User-Agent: curl/7.64.1 > Accept: */* > < HTTP/1.1 301 TLS Redirect < Date: Tue, 02 Jun 2020 12:41:48 GMT < Server: Varnish < X-Varnish: 10843471 < X-Cache: cp3058 int < X-Cache-Status: int-front < Server-Timing: cache;desc="int-front" < Set-Cookie: WMF-Last-Access=02-Jun-2020;Path=/;HttpOnly;secure;Expires=Sat, 04 Jul 2020 12:00:00 GMT < Set-Cookie: WMF-Last-Access-Global=02-Jun-2020;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Sat, 04 Jul 2020 12:00:00 GMT < X-Client-IP: 46.222.230.216 < Location: https://en.wikipedia.org/wiki/Main_Page < Content-Length: 0 < Connection: keep-alive < * Connection #0 to host en.wikipedia.org left intact * Closing connection 0 willikins:puppet vgutierrez$ curl -X POST http://en.wikipedia.org/wiki/Main_Page -v * Trying 91.198.174.192... * TCP_NODELAY set * Connected to en.wikipedia.org (91.198.174.192) port 80 (#0) > POST /wiki/Main_Page HTTP/1.1 > Host: en.wikipedia.org > User-Agent: curl/7.64.1 > Accept: */* > < HTTP/1.1 403 Insecure Request Forbidden - use HTTPS - https://lists.wikimedia.org/pipermail/mediawiki-api-announce/2016-May/000110.html < Date: Tue, 02 Jun 2020 12:41:53 GMT < Server: Varnish < X-Varnish: 78776135 < X-Cache: cp3058 int < X-Cache-Status: int-front < Server-Timing: cache;desc="int-front" < Set-Cookie: WMF-Last-Access=02-Jun-2020;Path=/;HttpOnly;secure;Expires=Sat, 04 Jul 2020 12:00:00 GMT < Set-Cookie: WMF-Last-Access-Global=02-Jun-2020;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Sat, 04 Jul 2020 12:00:00 GMT < X-Client-IP: 46.222.230.216 < Content-Type: text/html; charset=utf-8 < Content-Length: 1928 < Connection: keep-alive
We will have to add some Lua code to handle pybal checks to tls.lua too, something along these lines:
function do_global_read_request()
if ts.client_request.header['Host'] == 'varnishcheck.wikimedia.org' and ts.client_request.get_uri() == '/from/pybal' then
ts.http.intercept(function()
ts.say('HTTP/1.0 200 OK\r\n' ..
'Content-Type: text/plain\r\n' ..
'Content-Length: 0\r\n' ..
'Cache-Control: no-cache\r\n')
end)
end
endThe Host header used by pybal should also be changed from varnishcheck.wikimedia.org to something else which isn't software-specific like "healthecks.wikimedia.org" or whatever.