Tue, Jul 16
Mon, Jul 15
Fri, Jul 12
Thu, Jul 11
Wed, Jul 10
Note that VTC tests do not have to be run on cache servers anymore. They can be executed from SRE workstations using vagrant, which should help here. See https://wikitech.wikimedia.org/wiki/Varnish#Configuration
This is now done!
Tue, Jul 9
ATS now sets Server only if missing in the origin server response. Also, Varnish now does not send Via any longer (the header wasn't used at all).
Mon, Jul 8
All cache_upload nodes are now using ATS instead of Varnish for on-disk caching. Closing.
Fri, Jul 5
The host has been in production for weeks without issues now. Closing.
With the conversion of cp1090 this is now done.
@Gilles: is there anything left to be done here? Other than blogging about the results that is. :-)
Thu, Jul 4
The maximum allowed request header size (field name + value) is now 8192 bytes. Closing.
Wed, Jul 3
I've prepared https://gerrit.wikimedia.org/r/520425 to restrict the request/response headers sent to logstash by the various varnishlogconsumer daemons.
Tue, Jul 2
Mon, Jul 1
Fri, Jun 28
@Trizek-WMF: personally, I've tried and failed for days to reproduce the issue. My understanding is that occasionally some page loads for logged-in users take a very long time to complete, with characters slowly showing up on the screen.
I cannot reproduce the issue right now, but it does look like a strange interaction between the application servers and varnish.
Thu, Jun 27
Just bumped into another very frequent one:
Wed, Jun 26
Tue, Jun 25
Note that thumbor is occasionally returning 500 for that object. Hitting ATS to skip varnish-fe transformations:
@Gilles we've just had an interesting report on #wikimedia-operations that seems related.
Mon, Jun 24
We believe that Varnish fetch failures might be related to this issue, investigation is ongoing T226375