Upgrade all cache clusters to Varnish 4
Closed, ResolvedPublic

Description

The maps cache cluster has been upgraded to Varnish 4: https://phabricator.wikimedia.org/T122880.

We need to upgrade misc, upload and text.

Related Objects

StatusAssignedTask
DuplicateNone
DuplicateNone
OpenNone
OpenNone
DuplicateNikerabbit
Opendaniel
DuplicateBBlack
StalledNone
OpenNone
OpenNone
OpenNone
Resolvedema
DeclinedNone
Resolvedema
Resolvedema
Resolvedema
Resolvedema
Resolvedema
Resolvedema
Resolvedema
ResolvedBBlack
Resolvedema
Resolvedema
Resolvedema
ResolvedBBlack
Resolvedema
ema created this task.Apr 1 2016, 1:56 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 1 2016, 1:56 PM
ema claimed this task.
ema renamed this task from Uprade all cache clusters to Varnish 4 to Upgrade all cache clusters to Varnish 4.Apr 4 2016, 4:27 PM

Change 293296 had a related patch set uploaded (by Ema):
Add 'varnish_version' salt grain

https://gerrit.wikimedia.org/r/293296

Change 293296 merged by Ema:
Add 'varnish_version' salt grain

https://gerrit.wikimedia.org/r/293296

One of the goals for this quarter (Q4 2015-16) was to more-fully evaluate the difficulties for converting cache_text and cache_upload, and decide which we'd like to tackle next. The summary output of those evaluations is:

  • cache_text unique difficulties
    • Several applayer backends that may differ on behaviors for compatibility (MW/hhvm, restbase, cxserver, citoid)
    • In general, traffic is much more general-case: lots of distinct kinds of cacheability, conditional responses, etc.
    • geoip support - requires updating/writing new C-in-VCL or evaluating existing geoip2 vmods or writing a new one
    • Analytics here is much more important than any other cluster, and so far we've seen a pattern that our varnish4 analytics issues have an ongoing long-tail of bugfixing that's not complete yet.
  • cache_upload:
    • One backend to deal with for now (swift), with one kind of cacheability behavior.
    • Support for large objects in combination with streaming and Range is a somewhat unique problem to solve, but it seems likely varnish4 supports these things much better than varnish3 did in general. Some minor open questions about whether all of our old patched/plus functionality on these matters exists in varnish4, but again seems likely.

On the balance of these things, we selected cache_upload as the next target, to complete conversion during Q1 2016-17.

ema moved this task from Triage to Varnish v4 on the Traffic board.Sep 30 2016, 2:45 PM

Change 322252 had a related patch set uploaded (by Ema):
cache: get rid of varnish 3 compatibility code

https://gerrit.wikimedia.org/r/322252

Change 322252 merged by Ema:
cache: get rid of varnish 3 compatibility code

https://gerrit.wikimedia.org/r/322252

ema closed this task as Resolved.