We have a bunch of sites that are not fronted by the CDN. This also means they generally only live in eqiad and/or codfw, which also means that users in EMEA/Asia/etc take a totally different path to these sites than they do to the CDN.
Several times a year, some SREs and other technical contributors report being unable to reach sites like Gerrit, GitLab, lists.wm.o, integration.wm.o, icinga.wm.o, etc.
Let's make these sites also serve the same NEL headers as production.
Probably the best way to do this is to define some top-level hiera with a map of names and values for headers (we need to serve both NEL: and Report-To:) and then modify other Puppet code to reference this hiera as needed (whether Apache or Nginx config snippets).
We can also make the current gross mess that is in wikimedia-frontend.vcl.erb be generated from this hiera as well.
It's been almost five years since we deployed NEL on all CDN domains, and almost three years since this task was filed. In that time we've not changed the NEL configuration once. Copying and pasting is fine actually :)
To add NEL to your service
Set response headers like the following:
Report-To: {"group": "wm_nel", "max_age": 604800, "endpoints": [{"url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0"}]}
NEL: {"report_to": "wm_nel", "max_age": 604800, "failure_fraction": 0.05, "success_fraction": 0.0}You may wish to increase failure_fraction for low-traffic services, but 0.05 should always be a safe starting point even for high-traffic services: it's what we use on-wiki.
- gerrit
- lists
- integration
- icinga
- everything on WMCS / toolserver
-
gitlab