We currently serve a bunch of debug headers in all our responses, that are added or let through by Varnish and aren't necessary for page functionality.
While compressed, all these headers have a cost that could be avoided. I propose that we instead have the edge use a whitelist of acceptable headers. And if the x-wikimedia-debug header is present in the request, all response headers are let through.
I think it would be only a very small inconvenience for any developer trying to debug something to have to pass the x-wikimedia-debug header, either via the browser extension or the command line.
The edge cache can still cache custom headers, they would just be filtered out by the whitelist before being transmitted to the client.
Specifically, looking at a response from the text cache, I think the following could be removed:
backend-timing (could be passed via server-timing instead, which the client/NavTiming can collect)
x-analytics (possibly, it can't be read by the client, I assume it's only there for debugging purposes @analytics may be able to shed some light on that one)
x-cache-status (already available in the client-readable server-timing)
And for the upload cache, those additional ones:
If any of these are considered to be critically needed for all requests, I would suggest moving them to the server-timing header(s), which unlike other custom headers, a growing number of clients can read: https://www.chromestatus.com/feature/5695708376072192