Consider using 'Cache-Control: immutable' on versioned/long-cache load.php responses
Closed, ResolvedPublic
Actions

Description

Support landed in Firefox 49 (https://bugzilla.mozilla.org/show_bug.cgi?id=1267474).

In summary (from https://bitsup.blogspot.co.uk/2016/05/cache-control-immutable.html) the main benefit of "immutable" (compared to a high max-age) is that "immutable" is not bypassed when reloading a page.

Currently, regardless of max-age, browsers often ignore local cache expiry and perform If-None-Match/If-Modified-Since requests (yielding "304 Not Modified" responses for static assets). These are smaller than the "200 OK" response would have been (since browsers do remember and send E-Tag or Last-Modified headers), but are still worse than proper cache hits due to the required roundtrips for each 304 response.

Impact may be limited on Chrome since Chrome, unlike Firefox, does not ignore local cache expiry for subresources. When reloading a page in Chrome, cache-miss is only enforced on the main document. (per https://bugs.chromium.org/p/chromium/issues/detail?id=654378)

Background:

https://bitsup.blogspot.co.uk/2016/05/cache-control-immutable.html
https://www.ietf.org/mail-archive/web/httpbisa/current/msg25463.html
https://webplatformdaily.org/releases/2016-05-12
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
Firefox / Launch bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1267474
Chromium / Launch bug: https://bugs.chromium.org/p/chromium/issues/detail?id=611416
https://tools.ietf.org/html/rfc8246 HTTP Immutable Responses, September 2017

Details

	Subject	Repo	Branch	Lines +/-
	[WIP] resourceloader: Add 'Cache-control: immutable' to versioned responses	mediawiki/core	master	+7 -1
	static.php: Set "Cache-Control: immutable" on long-cache responses	operations/mediawiki-config	master	+2 -2

Customize query in gerrit

Related Objects

Mentioned Here: T229245: Reduce size of module version hash

Event Timeline

Krinkle created this task.Nov 2 2016, 7:36 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 2 2016, 7:36 PM

Krinkle triaged this task as Low priority.Nov 2 2016, 7:36 PM

Krinkle added a project: Performance-Team.

Krinkle moved this task from Inbox to Backlog on the MediaWiki-ResourceLoader board.

• Gilles moved this task from Inbox, needs triage to Backlog: Maintenance, non-prioritized on the Performance-Team board.Nov 3 2016, 9:54 PM

Krinkle renamed this task from Use 'Cache-Control: immutable' in versioned load.php response to Consider using 'Cache-Control: immutable' in versioned load.php response.Feb 9 2017, 10:22 PM

Krinkle moved this task from Backlog to Accepted Enhancement on the MediaWiki-ResourceLoader board.

Another write up from Mozilla, 1 year later:
https://hacks.mozilla.org/2017/01/using-immutable-caching-to-speed-up-the-web

And from Chromium:
https://blog.chromium.org/2017/01/reload-reloaded-faster-and-leaner-page_26.html

Facebook also wrote about it again in-depth. Also noting that, again, this mostly helps Firefox. Given that Chrome doesn't make these requests.
https://code.facebook.com/posts/557147474482256

Krinkle updated the task description. (Show Details)Mar 7 2017, 12:14 AM

Krinkle updated the task description. (Show Details)Mar 7 2017, 12:17 AM

Change 341483 had a related patch set uploaded (by krinkle):
[mediawiki/core] [WIP] resourceloader: Add 'Cache-control: immutable' to versioned responses

https://gerrit.wikimedia.org/r/341483

gerritbot added a project: Patch-For-Review.Mar 7 2017, 2:54 AM

Facebook Engineering Blog wrote

Once Chrome settled on a fix, we started engaging other browser vendors about the reload button behavior. We filed a bug with Firefox and they chose not to alter the long standing behavior of the reload button. Instead, Firefox implemented a proposal from one of our engineers to add a new cache-control header for some resources in order to tell the browser that this resource should never be revalidated.

As mentioned in various of the related bug reports and mailing lists, the proposed solution by Firefox is still suboptimal compared to Chrome's behaviour. Chrome only ignores cache for the main document (usually HTML) being reloaded, not the subresources. The subresources, after all, have valid cache expiry still. If they should be invalidated earlier, the natural way to do that is to either have shorter expiries, no-cache, or change the url if the resources are not meant to be compatible (e.g. version hash in path or query string).

Adding Cache-control: immutable would make Firefox no longer revalidate subresources with long expiries (> 30 days), but it would still needlessly revalidate the startup module and stylesheets, regardless of whether the intended expiry of 5-30min was reached. We can't add immutable to these responses.

Nemo_bis subscribed.Mar 9 2017, 3:18 PM

Facebook Engineering Blog wrote

Once Chrome settled on a fix, we started engaging other browser vendors about the reload button behavior. We filed a bug with Firefox and they chose not to alter the long standing behavior of the reload button. Instead, Firefox implemented a proposal from one of our engineers to add a new cache-control header for some resources in order to tell the browser that this resource should never be revalidated.

Safari has also been fixed to not revalidate unexpired subresources on reload, per https://bugs.webkit.org/show_bug.cgi?id=169756.

Change 341483 abandoned by Krinkle:
[WIP] resourceloader: Add 'Cache-control: immutable' to versioned responses

Reason:
Closing for now. See T149837 for more info. We may wanna reconsider this at some point (will leave the task open), but not actively being worked on. This patch could be re-used at that point.

https://gerrit.wikimedia.org/r/341483

Krinkle removed a project: Patch-For-Review.Mar 21 2017, 9:37 PM

Krinkle moved this task from Accepted Enhancement to Backlog on the MediaWiki-ResourceLoader board.

Krinkle moved this task from Backlog: Maintenance, non-prioritized to To-do: Goals, prioritized next 4 Quarters on the Performance-Team board.Aug 12 2018, 12:54 PM

Krinkle moved this task from To-do: Goals, prioritized next 4 Quarters to Backlog: Maintenance, non-prioritized on the Performance-Team board.Mar 14 2019, 5:35 PM

Krinkle added a project: patch-welcome.Jun 6 2019, 9:36 AM

Krinkle renamed this task from Consider using 'Cache-Control: immutable' in versioned load.php response to Consider using 'Cache-Control: immutable' on versioned/long-cache load.php responses.Sep 19 2019, 8:58 PM

Change 540963 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[operations/mediawiki-config@master] static.php: Set "Cache-Control: immutable" on long-cache responses

https://gerrit.wikimedia.org/r/540963

gerritbot added a project: Patch-For-Review.Oct 4 2019, 8:25 PM

Change 540963 merged by jenkins-bot:
[operations/mediawiki-config@master] static.php: Set "Cache-Control: immutable" on long-cache responses

https://gerrit.wikimedia.org/r/540963

Change 341483 restored by Krinkle:
[WIP] resourceloader: Add 'Cache-control: immutable' to versioned responses

https://gerrit.wikimedia.org/r/341483

Krinkle removed a project: patch-welcome.Oct 4 2019, 8:55 PM

Fomafix updated the task description. (Show Details)Oct 10 2019, 9:46 AM

Fomafix subscribed.

Krinkle removed a project: Patch-For-Review.Mar 27 2020, 4:57 PM

• Gilles added a project: EngProd-Virtual-Hackathon.May 20 2020, 5:23 PM

My hesitation here stems from the fact that we've reduced our version hash size (ref T229245, and blog post), and we've drastically reduced the internal CDN expiry at WMF (from 30 days to 1 day, Wikitech docs), and we proactively invalidate client-side module storage every month (T229245).

All that to say: we already tell browsers to unconditionally and offline use module responses for 30 days as the Cache-Control max-age value declares. The only thing we're looking to gain from this task is to fix the Firefox behaviour where a reload also re-downloads static assets for no reason.

Scope

This apparent inefficiency is limited to reloading the same page, and specifically limited to Firefox.

Chrome and Safari have already treated that behaviour as a bug and fixed it. And setting immutable on all static resources has the side-effect of making the max-age ineffective. It is our expectation that after the max-age expires, we get to renew the response. This allows propagation of untracked changes such as OS/PHP upgrades, site configuration changes, and anything else that might not propagate on its own through ParserCache and client-side caches otherwise within a set period of time. We also need to consider the constrains of our version hashes and what they are designed for. The 5-character FNV hash was chosen for its speed and its ability to reliabily detect change within a given module's recent history (30 days). It is not designed to uniquely identify arbitrary code for years. We'd have to transfer massive SHA256 hashes to support that, or otherwise adopt a globally unique naming scheme for every deployment. This works well for individually served libraries and fonts with proper names and sequentially versioned releases like lib/jquery-3.5.0.js.

On the bright side for Firefox, our client-side module storage (RL Architecture docs) means that we aren't actually affected by this issue much in the first place since we don't generally fetch cached module resources from the browser's HTTP cache. We fetch them from localStorage instead, which isn't impacted by reloads. We control this cache access directly and they work just as well on reloads as regular page loads.

Exceptions where we do rely on browser cache:

RL startup module, this doesn't qualify for immutable, given it's short-lived 5 minute nature.
static image assets, such as site logos. These are served outside ResourceLoader, and I already treated those with Cache-Control: immutable via wmf-config/static.php (source).

As per the original task title - I'll close this resolved since we considered it, and applied it where it makes sense and is safe to do.

Change 341483 abandoned by Krinkle:

[mediawiki/core@master] [WIP] resourceloader: Add 'Cache-control: immutable' to versioned responses

Reason:

https://gerrit.wikimedia.org/r/341483

Krinkle moved this task from Backlog: Maintenance, non-prioritized to To-do: Goals prioritized current Quarter on the Performance-Team board.Sep 8 2022, 2:19 PM

Krinkle moved this task from Untriaged to Ready for write-up on the Wikimedia-Performance-publish board.Sep 10 2022, 12:14 AM

Restricted Application added a project: MediaWiki-Platform-Team. · View Herald TranscriptAug 26 2023, 8:32 PM

Consider using 'Cache-Control: immutable' on versioned/long-cache load.php responsesClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

Scope

Consider using 'Cache-Control: immutable' on versioned/long-cache load.php responses
Closed, ResolvedPublic
Actions