Page MenuHomePhabricator

Reduce amount of headers sent from web responses
Closed, ResolvedPublic

Description

Some of the response headers we are sending to clients serve no actual purpose and we could strip them out at the frontend layer.

HeaderExampleStatus
X-Powered-ByPHP/7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1 Debug only, see T210484
X-Varnish521726689 533337780, 225083667 220092282, 525815818 515121340 Debug only, see T210484
Servermw1238.eqiad.wmnet To be kept
Via1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1) Removed
X-Analyticshttps=1;nocookies=1 Removed

In the specific case of cache_upload, there are a few headers set by swift which we might also consider removing:

HeaderExample
X-Object-Meta-Sha1Base36iui1kxpdsmxapvmrdyhn5vaxn2il53x
X-Timestamp1487337414.04422
X-Trans-Idtx75bd096fde6d4999bf946-005afabc1f

See also:

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ema triaged this task as Medium priority.May 16 2018, 10:18 AM
ema moved this task from Triage to Caching on the Traffic board.

We could also simply avoid X-Powered-By at the source; our PHP configs already use "expose_php=off" and for HHVM per https://github.com/facebook/hhvm/issues/2343 adding "expose_php = 0" to server.ini would be the HHVM equivalent.

The X-Powered-By part is actually useful for us in order to discern the source of rendering of a page - be it hhvm or php.

We will use it during the HHVM => PHP7 migration, so let's not filter it for now.

Ditto for some Thumbor headers:

thumbor-engine: wikimedia_thumbor.engine.imagemagick
thumbor-processing-time: 413
thumbor-processing-utime: 316
thumbor-request-date: Wed, 16 May 2018 10:30:38 GMT
thumbor-request-id: 9647181157a7e9437a3a317dce66ce96

Change 433573 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] vcl: strip away unnecessary response headers set by Thumbor

https://gerrit.wikimedia.org/r/433573

X-Varnish	521726689 533337780, 225083667 220092282, 525815818 515121340
Server	mw1238.eqiad.wmnet

Especially these two can be handy. In the past I've used them to quickly help identify everything from varnish servers that got cut of from the cache invalidation streams, to accidentally pooled servers which were running outdated or incorrectly configured setups.

Now if we are so confident that we can guarantee other detection methods for such problems, than i'm good, but wanted to put it out there.

Change 433573 merged by Ema:
[operations/puppet@production] vcl: strip away unnecessary response headers set by Thumbor

https://gerrit.wikimedia.org/r/433573

Krinkle renamed this task from Remove unnecessary response headers to Reduce amount of headers sent from Varnish responses.Jun 6 2018, 5:57 PM
Krinkle renamed this task from Reduce amount of headers sent from Varnish responses to Reduce amount of headers sent from web responses.
Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.

@ema Could we use std.log (VCL_Log) to report X-Analytics data and stop the header from reaching the final user?

Or (maybe crazy thought) remove the headers entirely at the nginx layer while varnish work of reorganizing headers is taking place?

Vvjjkkii renamed this task from Reduce amount of headers sent from web responses to tvcaaaaaaa.Jul 1 2018, 1:10 AM
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
Tbayer renamed this task from tvcaaaaaaa to Reduce amount of headers sent from web responses.Jul 1 2018, 2:42 PM
Tbayer lowered the priority of this task from High to Medium.
Tbayer updated the task description. (Show Details)
Tbayer added subscribers: GerritBot, Aklapper.

Change 521261 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] varnish: stop sending the Via response header

https://gerrit.wikimedia.org/r/521261

Change 521261 merged by Ema:
[operations/puppet@production] varnish: stop sending the Via response header

https://gerrit.wikimedia.org/r/521261

Change 583570 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: remove debug HTTP headers if X-Wikimedia-Debug is absent

https://gerrit.wikimedia.org/r/583570

Change 583942 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] cache: stop sending X-Varnish

https://gerrit.wikimedia.org/r/583942

Change 583570 merged by Ema:
[operations/puppet@production] ATS: unset debug HTTP headers for normal requests

https://gerrit.wikimedia.org/r/583570

Change 583942 merged by Ema:
[operations/puppet@production] cache: stop sending X-Varnish

https://gerrit.wikimedia.org/r/583942

ema updated the task description. (Show Details)