Page MenuHomePhabricator

Find out if Varnish is messing with ETags, and what to do about it.
Closed, ResolvedPublic

Description

It appears that Varnish is "weakening" ETags when it applies on-the-fly compression. This prevents ETags from working with If-Match headers, causing the If-match to fail with a 412 for all requests.

Is this actually the case for the WMF setup? The context is T238849 and T310710. This is a blocked for the migration of VE endpoints away from RESTbase (T264669).

Short summary of the pickle we are in:

  • Visual Editor wants to use If-Match to the parsoid transform endpoint, to ensure that the transformation is performed based on the correct rendering, to avoid dirty diffs.
  • Visual Editor uses the etag it received from the server. This observed to be a weak etag on requests like https://en.wikipedia.org/api/rest_v1/page/html/Earth.
  • RESTbase correctly emits strong ETags.
  • RESTbase incorrectly allows weak ETags in If-Match headers. MediaWiki's REST framework doesn't.
  • Presumably, Varnish applies on-the-fly compression on the response sent by RESTbase.
  • When applying compression, Varnish will set the Content-Encoding header, and "weaken" the ETag.
  • HTTP requires different (strong) ETags to be used for different Content-Encodings.
  • Using Transfer-Encoding to indicate compression would allow us to keep the ETag intact. But apparently, this isn't widely supported in clients, and Varnish doesn't seem to offer it either.

There seems to be no way to have on-the-fly compression in a way that allows clients to use If-Modified. Is that correct? Is Varnish really at fault here? And can anything be donme about that?