Page MenuHomePhabricator

Serve some default well known files for Toolforge webservices
Closed, ResolvedPublic

Description

Before the introduction of the 2020 Kubernetes ingress layer, the admin tool and/or the urlproxy front proxy provided some well-known urls like /favicon.ico and /robots.txt for the tools.wmflabs.org domain. The new setup allows tools to provide these themselves, which I think is a good thing, but it also currently does not step in to provide a reasonable default if the tool does not handle these routes.

We should figure out where we can intercept 404 responses for at least /favicon.ico and /robots.txt and return sensible defaults.

Event Timeline

I have a hunch that this needs to be done all the way up in urlproxy to be effective across grid+k8s and tools.wmflabs.org+*.toolforge.org

bd808 raised the priority of this task from Low to Medium.May 5 2020, 4:38 PM

Change 622237 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] dynamicproxy: serve default /robots.txt and /favicon.ico for Toolforge

https://gerrit.wikimedia.org/r/622237

Change 622237 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] dynamicproxy: serve default /robots.txt and /favicon.ico for Toolforge

https://gerrit.wikimedia.org/r/622237

Mentioned in SAL (#wikimedia-cloud) [2020-08-26T21:08:00Z] <bd808> Disabled puppet on tools-proxy-06 to test fixes for a bug in the new T251628 code

Change 622677 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] dynamicproxy: un-nest /favicon.ico and /robots.txt locations

https://gerrit.wikimedia.org/r/622677

Change 622677 merged by Bstorm:
[operations/puppet@production] dynamicproxy: un-nest /favicon.ico and /robots.txt locations

https://gerrit.wikimedia.org/r/622677

It took a couple of tries (like every nginx config change :/) but its alive! Only thing left to do is tell cloud-announce because of the behavior change in robots.txt.

Announced on cloud-announce@ and documented in the Toolforge webservice help docs on wikitech.