Maniphest T194814

Reduce amount of headers sent from web responses
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	• ema
	May 16 2018, 10:18 AM

Description

Some of the response headers we are sending to clients serve no actual purpose and we could strip them out at the frontend layer.

Header	Example	Status
X-Powered-By	PHP/7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1	Debug only, see T210484
X-Varnish	521726689 533337780, 225083667 220092282, 525815818 515121340	Debug only, see T210484
Server	mw1238.eqiad.wmnet	To be kept
~~Via~~	~~1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1)~~	Removed
~~X-Analytics~~	~~https=1;nocookies=1~~	Removed

In the specific case of cache_upload, there are a few headers set by swift which we might also consider removing:

Header	Example
X-Object-Meta-Sha1Base36	iui1kxpdsmxapvmrdyhn5vaxn2il53x
X-Timestamp	1487337414.04422
X-Trans-Id	tx75bd096fde6d4999bf946-005afabc1f

Details

Subject	Repo	Branch	Lines +/-
cache: stop sending X-Varnish	operations/puppet	production	+4 -6
ATS: unset debug HTTP headers for normal requests	operations/puppet	production	+23 -0
varnish: stop sending the Via response header	operations/puppet	production	+1 -1
vcl: strip away unnecessary response headers set by Thumbor	operations/puppet	production	+36 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T194814 Reduce amount of headers sent from web responses
Resolved	• Nuria	T196558 Send X-Analytics information from Varnish to Hadoop with VCL_Log
		Restricted Task
Resolved	• Gilles	T210484 Only serve debug HTTP headers when x-wikimedia-debug is present

Event Timeline

• ema created this task.May 16 2018, 10:18 AM

Restricted Application added a project: SRE. · View Herald TranscriptMay 16 2018, 10:18 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

• ema triaged this task as Medium priority.May 16 2018, 10:18 AM

• ema moved this task from Backlog to Caching on the Traffic board.

We could also simply avoid X-Powered-By at the source; our PHP configs already use "expose_php=off" and for HHVM per https://github.com/facebook/hhvm/issues/2343 adding "expose_php = 0" to server.ini would be the HHVM equivalent.

The X-Powered-By part is actually useful for us in order to discern the source of rendering of a page - be it hhvm or php.

We will use it during the HHVM => PHP7 migration, so let's not filter it for now.

Ditto for some Thumbor headers:

thumbor-engine: wikimedia_thumbor.engine.imagemagick
thumbor-processing-time: 413
thumbor-processing-utime: 316
thumbor-request-date: Wed, 16 May 2018 10:30:38 GMT
thumbor-request-id: 9647181157a7e9437a3a317dce66ce96

Change 433573 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] vcl: strip away unnecessary response headers set by Thumbor

https://gerrit.wikimedia.org/r/433573

gerritbot added a project: Patch-For-Review.May 17 2018, 2:16 PM

• ema updated the task description. (Show Details)May 17 2018, 2:17 PM

X-Varnish	521726689 533337780, 225083667 220092282, 525815818 515121340
Server	mw1238.eqiad.wmnet

Especially these two can be handy. In the past I've used them to quickly help identify everything from varnish servers that got cut of from the cache invalidation streams, to accidentally pooled servers which were running outdated or incorrectly configured setups.

Now if we are so confident that we can guarantee other detection methods for such problems, than i'm good, but wanted to put it out there.

Change 433573 merged by Ema:
[operations/puppet@production] vcl: strip away unnecessary response headers set by Thumbor

https://gerrit.wikimedia.org/r/433573

Krinkle added a project: Performance-Team (Radar).Jun 6 2018, 5:51 PM

Krinkle updated the task description. (Show Details)

Krinkle mentioned this in T110353: Audit use of cookies.

Krinkle added a subtask: T196558: Send X-Analytics information from Varnish to Hadoop with VCL_Log.Jun 6 2018, 5:54 PM

Krinkle renamed this task from Remove unnecessary response headers to Reduce amount of headers sent from Varnish responses.Jun 6 2018, 5:57 PM

Krinkle renamed this task from Reduce amount of headers sent from Varnish responses to Reduce amount of headers sent from web responses.

Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.

@ema Could we use std.log (VCL_Log) to report X-Analytics data and stop the header from reaching the final user?

Or (maybe crazy thought) remove the headers entirely at the nginx layer while varnish work of reorganizing headers is taking place?

• Vvjjkkii renamed this task from Reduce amount of headers sent from web responses to tvcaaaaaaa.Jul 1 2018, 1:10 AM

• Vvjjkkii raised the priority of this task from Medium to High.

• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).

• Vvjjkkii updated the task description. (Show Details)

• Vvjjkkii removed subscribers: gerritbot, Aklapper.

• Tbayer renamed this task from tvcaaaaaaa to Reduce amount of headers sent from web responses.Jul 1 2018, 2:42 PM

• Tbayer lowered the priority of this task from High to Medium.

• Tbayer removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.

• Tbayer updated the task description. (Show Details)

• Tbayer added subscribers: GerritBot, Aklapper.

Krinkle updated the task description. (Show Details)Jul 1 2018, 9:51 PM

Krinkle mentioned this in T196558: Send X-Analytics information from Varnish to Hadoop with VCL_Log.Nov 27 2018, 7:49 PM

Krinkle mentioned this in T210484: Only serve debug HTTP headers when x-wikimedia-debug is present.

Krinkle removed a project: Patch-For-Review.Nov 27 2018, 7:52 PM

Krinkle moved this task from Watching to Perf recommendation on the Performance-Team (Radar) board.Jan 21 2019, 1:08 AM

• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 10:01 PM

Change 521261 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] varnish: stop sending the Via response header

https://gerrit.wikimedia.org/r/521261

gerritbot added a project: Patch-For-Review.Jul 8 2019, 12:39 PM

Change 521261 merged by Ema:
[operations/puppet@production] varnish: stop sending the Via response header

https://gerrit.wikimedia.org/r/521261

Krinkle removed a project: Patch-For-Review.Aug 5 2019, 10:28 PM

Krinkle moved this task from Radar to To-do: Goals, prioritized next 4 Quarters on the Performance-Team board.Sep 15 2019, 2:02 AM

Krinkle edited projects, added Performance-Team; removed Performance-Team (Radar).

Krinkle updated the task description. (Show Details)

Krinkle moved this task from To-do: Goals, prioritized next 4 Quarters to Radar on the Performance-Team board.Sep 15 2019, 2:07 AM

Krinkle edited projects, added Performance-Team (Radar); removed Performance-Team.

Ladsgroup subscribed.Sep 16 2019, 10:18 PM

Krinkle added parent tasks: T210484: Only serve debug HTTP headers when x-wikimedia-debug is present, Restricted Task.Feb 10 2020, 11:59 PM

Krinkle removed parent tasks: Restricted Task, T210484: Only serve debug HTTP headers when x-wikimedia-debug is present.

Krinkle added subtasks: Restricted Task, T210484: Only serve debug HTTP headers when x-wikimedia-debug is present.

Krinkle closed subtask T196558: Send X-Analytics information from Varnish to Hadoop with VCL_Log as Resolved.Mar 9 2020, 6:16 PM

Krinkle updated the task description. (Show Details)Mar 9 2020, 6:19 PM

Aklapper removed a subscriber: GerritBot.Mar 22 2020, 9:32 PM

Change 583570 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: remove debug HTTP headers if X-Wikimedia-Debug is absent

https://gerrit.wikimedia.org/r/583570

gerritbot added a project: Patch-For-Review.Mar 26 2020, 10:40 AM

Change 583942 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] cache: stop sending X-Varnish

https://gerrit.wikimedia.org/r/583942

Change 583570 merged by Ema:
[operations/puppet@production] ATS: unset debug HTTP headers for normal requests

https://gerrit.wikimedia.org/r/583570

Change 583942 merged by Ema:
[operations/puppet@production] cache: stop sending X-Varnish

https://gerrit.wikimedia.org/r/583942

• ema closed subtask T210484: Only serve debug HTTP headers when x-wikimedia-debug is present as Resolved.Apr 1 2020, 2:00 PM

• ema closed this task as Resolved.Apr 1 2020, 2:04 PM

• ema updated the task description. (Show Details)

Krinkle closed subtask Restricted Task as Resolved.Aug 6 2023, 10:25 PM