Port remaining scripts depending on varnishlog.py to new VSL API
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• ema
	Mar 31 2016, 1:50 PM

Description

We've ported varnishreqstats and varnishstatsd in https://phabricator.wikimedia.org/T128788.

The following scripts are left to be ported:

Phase 1: Scripts needed for text role

Script	Purpose
modules/varnish/files/varnishrls	Tracks stats on ResourceLoader (load.php) requests.
modules/varnish/files/varnishxcps	Reports protocol and cipher support stats based on X-Connection-Properties header.

Phase 2: Scripts needed for upload role

Script	Purpose
modules/varnish/files/varnishmedia	Tracks stats on image requests.
modules/varnish/files/varnishprocessor	Library, currently only used by varnishmedia.

Details

Subject	Repo	Branch	Lines +/-
varnishrls4: use VSL query and proper tags	operations/puppet	production	+14 -15
Port varnishrls to Varnish 4 VSL API	operations/puppet	production	+90 -1
Port varnishmedia to new VSL API	operations/puppet	production	+67 -1
Port varnishprocessor to new VSL API	operations/puppet	production	+117 -1
varnishlog4: allow methods to be used as callbacks	operations/puppet	production	+9 -2
Port varnishxcps to new VSL API	operations/puppet	production	+103 -1
varnish+statsd: refactor classes, move rls to text-only	operations/puppet	production	+16 -29

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
		Restricted Task
Duplicate	None	T109331 Deleted files sometimes remain visible to non-privileged users if permanently linked
Duplicate	None	T133819 upload-lb.ulsfo.wikimedia.org still allow access to some deleted files
Open	None	T124286 [Epic] Wikidata language support
Open	None	T134592 Allow setting the UI to a language other than English for anonymous users
Duplicate	Nikerabbit	T149419 Interface language selection for unregistered users on Wikimedia projects
Invalid	None	T114662 RFC: Per-language URLs for multilingual wiki pages
Duplicate	BBlack	T119038 Image cache issue when 'over-writing' an image on commons
Resolved	• ema	T133821 Make CDN purges reliable
Resolved	• ema	T108580 HTTPS for internal service traffic
Invalid	None	T109325 Outbound HTTPS for varnish backend instances
Open	None	T122867 Evaluate the feasibility of cache invalidation for the action API
Resolved	• ema	T122881 Install XKey vmod
Declined	None	T142841 Sideways Only-If-Cached on misses at a primary DC
Resolved	• ema	T131499 Upgrade all cache clusters to Varnish 4
Resolved	• ema	T131353 Port remaining scripts depending on varnishlog.py to new VSL API

Event Timeline

• ema created this task.Mar 31 2016, 1:50 PM

Restricted Application added a project: SRE. · View Herald TranscriptMar 31 2016, 1:50 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

• ema triaged this task as Medium priority.Mar 31 2016, 1:50 PM

• ema moved this task from Backlog to Upcoming on the Traffic board.Apr 1 2016, 12:35 PM

• ema added a parent task: T131499: Upgrade all cache clusters to Varnish 4.Apr 1 2016, 1:57 PM

Change 281439 had a related patch set uploaded (by BBlack):
varnish statsd: refactor classes, move rls to text-only

https://gerrit.wikimedia.org/r/281439

gerritbot added a project: Patch-For-Review.Apr 4 2016, 1:50 PM

Change 281439 merged by BBlack:
varnish statsd: refactor classes, move rls to text-only

https://gerrit.wikimedia.org/r/281439

Change 282887 had a related patch set uploaded (by Ema):
Port varnishxcps to new VSL API

https://gerrit.wikimedia.org/r/282887

Change 282887 merged by Ema:
Port varnishxcps to new VSL API

https://gerrit.wikimedia.org/r/282887

Change 295207 had a related patch set uploaded (by Ema):
Port varnishrls to new VSL API

https://gerrit.wikimedia.org/r/295207

• ema mentioned this in rOPUP7868f709b07f: Port varnishrls to new VSL API.Jun 20 2016, 10:30 AM

• ema mentioned this in rOPUPcee60ec8f57b: Port varnishrls to new VSL API.

Change 305517 had a related patch set uploaded (by Ema):
varnishlog4: allow methods to be used as callbacks

https://gerrit.wikimedia.org/r/305517

• ema mentioned this in rOPUP09f19c68dde3: varnishlog4: allow methods to be used as callbacks.Aug 18 2016, 3:22 PM

Change 305525 had a related patch set uploaded (by Ema):
Port varnishprocessor to new VSL API

https://gerrit.wikimedia.org/r/305525

• ema mentioned this in rOPUP372ac0c17a9b: varnishlog4: allow methods to be used as callbacks.Aug 18 2016, 4:21 PM

• ema mentioned this in rOPUPc3a3fb91f117: Port varnishprocessor to new VSL API.

Change 305527 had a related patch set uploaded (by Ema):
Port varnishmedia to new VSL API

https://gerrit.wikimedia.org/r/305527

• ema mentioned this in rOPUP5bec0afd63ce: varnishlog4: allow methods to be used as callbacks.Aug 18 2016, 4:26 PM

• ema mentioned this in rOPUP219462f2d5be: Port varnishprocessor to new VSL API.

• ema mentioned this in rOPUP6aa9a4f8a430: Port varnishmedia to new VSL API.

Change 305517 merged by Ema:
varnishlog4: allow methods to be used as callbacks

https://gerrit.wikimedia.org/r/305517

• ema mentioned this in rOPUP58f87bd8a8c1: varnishlog4: allow methods to be used as callbacks.Aug 19 2016, 8:54 AM

Change 305525 merged by Ema:
Port varnishprocessor to new VSL API

https://gerrit.wikimedia.org/r/305525

Change 305527 merged by Ema:
Port varnishmedia to new VSL API

https://gerrit.wikimedia.org/r/305527

• ema mentioned this in rOPUP9446653938f6: Port varnishprocessor to new VSL API.Aug 19 2016, 9:14 AM

• ema mentioned this in rOPUPf090f2fed4aa: Port varnishmedia to new VSL API.

BBlack mentioned this in T131502: Convert upload cluster to Varnish 4.Aug 19 2016, 4:32 PM

BBlack moved this task from Upcoming to Backlog on the Traffic board.Sep 30 2016, 1:19 PM

• ema moved this task from Backlog to Varnish v4 on the Traffic board.Sep 30 2016, 2:43 PM

Change 295207 merged by Ema:
Port varnishrls to Varnish 4 VSL API

https://gerrit.wikimedia.org/r/295207

Change 316966 had a related patch set uploaded (by Ema):
varnishrls4: use VSL query and proper tags

https://gerrit.wikimedia.org/r/316966

Change 316966 merged by Ema:
varnishrls4: use VSL query and proper tags