Page MenuHomePhabricator

Move tools-static.wmflabs.org behind project-proxy
Closed, ResolvedPublic

Details

Related Objects

StatusAssignedTask
ResolvedKrenair

Event Timeline

Krenair created this task.Wed, Oct 30, 8:17 PM
Restricted Application added a project: Traffic. · View Herald TranscriptWed, Oct 30, 8:17 PM
Krenair claimed this task.Thu, Oct 31, 12:30 AM
Krenair added a subscriber: aborrero.

Change 547360 had a related patch set uploaded (by Alex Monk; owner: Alex Monk):
[operations/puppet@production] tools-static: Allow X-Forwarded-Proto: https header

https://gerrit.wikimedia.org/r/547360

Change 547363 had a related patch set uploaded (by Alex Monk; owner: Alex Monk):
[operations/puppet@production] toolforge: Remove direct TLS termination support from static-server

https://gerrit.wikimedia.org/r/547363

Change 547364 had a related patch set uploaded (by Alex Monk; owner: Alex Monk):
[operations/puppet@production] toolforge: Remove old absented star.wmflabs.org certificate

https://gerrit.wikimedia.org/r/547364

Krenair added a comment.EditedThu, Oct 31, 12:40 AM

Proposed plan here:

  • https://gerrit.wikimedia.org/r/547360
  • Check it accepts traffic properly with something like curl -H 'Host: tools-static.wmflabs.org' -H 'X-Forwarded-Proto: https' http://tools-static-12/pagecounts/pagecounts.json
  • Create proxy entry for tools-static.wmflabs.org pointing at this instance (manually if we have to, don't want to interrupt existing traffic)
  • If we had to do the previous step manually, update A record for tools-static to point at project-proxy. Wait for DNS TTL.
  • Traffic should still be working externally, test that with something like https://tools-static.wmflabs.org/pagecounts/pagecounts.json - logs should appear on the tools-static instance appearing from the proxy-01.project-proxy.eqiad.wmflabs internal IP
  • https://gerrit.wikimedia.org/r/547363
  • Wait for a puppet run
  • Confirm everything is fine
  • https://gerrit.wikimedia.org/r/547364
  • Disassociate floating IP

Change 547360 merged by Andrew Bogott:
[operations/puppet@production] tools-static: Allow X-Forwarded-Proto: https header

https://gerrit.wikimedia.org/r/547360

It's accepting traffic with X-Forwarded-Proto correctly, confirmed we have to do the proxy config manually (can't just add a new proxy through the horizon UI and have it take over the existing DNS A record), so I've configured the proxy with this on proxy-01.project-proxy.eqiad.wmflabs:

>>> import requests
>>> resp = requests.put(
...     'http://proxy-01.project-proxy.eqiad.wmflabs:5668/v1/tools/mapping',
...     json={
...         "backends": ['http://172.16.0.186:80'],
...         "domain": 'tools-static.wmflabs.org'
...     }
... )
>>> resp.content
b''
>>> resp.status_code
200

project-proxy is handling requests for it properly, e.g. with curl -H 'Host: tools-static.wmflabs.org' https://proxy-eqiad.wmflabs.org/pagecounts/pagecounts.json -v.

Next step is updating tools-static.wmflabs.org A record under wmflabsdotorg project to point at dynamicproxy.

Mentioned in SAL (#wikimedia-cloud) [2019-11-01T20:38:40Z] <Krenair> Updated A record for tools-static.wmflabs.org to point towards project-proxy T236952

Change 547363 merged by Andrew Bogott:
[operations/puppet@production] toolforge: Remove direct TLS termination support from static-server

https://gerrit.wikimedia.org/r/547363

Change 548861 had a related patch set uploaded (by Alex Monk; owner: Alex Monk):
[operations/puppet@production] tools-static: Fix old cert absent line

https://gerrit.wikimedia.org/r/548861

Change 548861 merged by Andrew Bogott:
[operations/puppet@production] tools-static: Fix old cert absent line

https://gerrit.wikimedia.org/r/548861

Change 547364 merged by Bstorm:
[operations/puppet@production] toolforge: Remove old absented star.wmflabs.org certificate

https://gerrit.wikimedia.org/r/547364

Mentioned in SAL (#wikimedia-cloud) [2019-11-05T22:49:54Z] <Krenair> Disassociated floating IP 185.15.56.60 from tools-static-13, traffic to this host goes via the project-proxy now. DNS was already changed a few days ago. T236952

Krenair closed this task as Resolved.Tue, Nov 5, 10:51 PM
bd808 awarded a token.Tue, Nov 5, 10:51 PM