Page MenuHomePhabricator

Image urls in CSS remain cached with old $wgResourceBasePath
Closed, ResolvedPublic

Description

From closed unrelated ticket: https://phabricator.wikimedia.org/T134332#2262358

Probably completely unrelated, but while looking at commons root URL in Chrome, I noticed a pair of 404s:

https://commons.wikimedia.org/static/1.27.0-wmf.8/resources/lib/jquery.ui/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png?944bb
https://commons.wikimedia.org/static/1.27.0-wmf.8/resources/lib/jquery.ui/themes/smoothness/images/ui-icons_888888_256x240.png?302ae

Both of these show up in chrome's network timeline as 404s referenced by onloadwff.js....

I still see these in a fresh load this morning 404-ing. What's the cause here?

Event Timeline

BBlack created this task.May 4 2016, 11:36 AM
Restricted Application added a project: Operations. · View Herald TranscriptMay 4 2016, 11:36 AM
Restricted Application added subscribers: Zppix, Steinsplitter, Aklapper. · View Herald Transcript
chasemp triaged this task as High priority.May 5 2016, 8:47 PM
BBlack added a comment.May 9 2016, 6:51 PM

Ping. Still seeing these 404s on https://commons.wikimedia.org/ . ...

I can't reproduce this. Wikimedia app servers haven't produced /static/1.27.0-$version/ urls since February 26 (config diff). Since then, we only use /w/ for static urls. (E.g. /w/resources.)

When opening pages on https://commons.wikimedia.org/ I see the following urls requests in the background:

The last wmf branch that used static symlinks was 1.27.0-wmf.15. We are currently at 1.27.0-wmf.23.

The (then unused) logic for branch-name static urls was removed in 3f2943e482. The remaining symlinks were removed in 29b8e505b22.

The references for jQuery UI urls are not cached in HTML, but in stylesheets served from load.php.

My guess is that these are perhaps cached client-side in your localStorage, and the module has not changed since February, and we are versioning the unminified styles (and as such, don't see the change in expanded urls when version hashing the for module).

Krinkle renamed this task from commons.wikimedia.org home page has 404s loaded from JS (RL?) to Image urls in CSS remain cached with old $wgResourceBasePath.May 9 2016, 7:22 PM

While we waited the usual 30 days before removing the symlinks for old branches, they were still cached client side (which has no expiry).

This is not a new problem and not caused by the migration to w/static.php. In the past when removing those symlinks they will broken cached urls as well. If anything, the migration to static.php solved this problem going forward as we now produce branch-agonistic urls that relate to file content only (and as long as a module doesn't change, its files don't change and remain accessible). Once an image changes, the module version changes, and produce a new url with the old one only staying around for as long as the startup module is cached (~5 minutes).

BBlack added a comment.May 9 2016, 7:32 PM

Ok. I can see that they're gone in a fresh browser install or incognito view. But I've definitely closed the whole browser app since the change, and no amount of force-reload gets rid of these 404s. I guess the links exist in something cached browser-side which a force-reload of the page does not actually reload?

Ok. I can see that they're gone in a fresh browser install or incognito view. But I've definitely closed the whole browser app since the change, and no amount of force-reload gets rid of these 404s. I guess the links exist in something cached browser-side which a force-reload of the page does not actually reload?

Indeed. In the most optimal scenario these are not HTTP requests, they are gets to a client-side key/value store, which are somewhat comparable to reading a cookie. Reloads do not affect them.

See https://www.mediawiki.org/wiki/ResourceLoader/Features#Store for more information about this feature.

BBlack added a comment.May 9 2016, 8:21 PM

So, how do we get rid of those stored broken URIs? It seems like a lot of clients will be stuck with them for a very long time...

The proper solution is to roll out pure module-content based versioning to file modules (which covers jquery-ui). More specifically, to apply hashing to module content post-minification, rather than before.

This requires careful auditing of the involved hot code paths. Several of these are not fast enough to trigger for all modules in the startup response (which is where version hashed are computed). Enabling this means we'd build (transform+concat) and minify over 1000 modules in https://en.wikipedia.org/w/load.php?debug=true&modules=startup&only=scripts. I believe this is feasible with the right optimisations and object caching in place, but we're not there yet.

The original ResourceLoader design had no content-based versioning (only timestamp-based versioning with the original files as input, no run-time configuration or transformations like CSSMin embedding and url expanding salted into the hash).

Last year I added content-based versioning, but this uncovered various performance issues in the Less compilation stack. So it's only enabled for "simple" modules that generate code (e.g. ResourceLoader LanguageDataModule). The main corpus of file modules don't have this enabled yet.

...

Short-term workaround: Increment $wgResourceLoaderStorageVersion in wmf-config.

Krinkle claimed this task.Jun 6 2016, 6:49 PM
Krinkle moved this task from Inbox to Doing on the Performance-Team board.

Mentioned in SAL [2016-06-08T22:01:32Z] <krinkle@tin> Synchronized wmf-config/CommonSettings.php: Bump wgResourceLoaderStorageVersion (T134368) (duration: 00m 28s)

Krinkle lowered the priority of this task from High to Medium.Jun 9 2016, 7:06 PM
Krinkle moved this task from Doing to Backlog: Small & Maintenance on the Performance-Team board.
Krinkle moved this task from Assigned to Backlog on the MediaWiki-ResourceLoader board.
Krinkle removed projects: Traffic, Operations.
Krinkle removed Krinkle as the assignee of this task.Jul 16 2016, 1:26 AM
Gilles closed this task as Resolved.Dec 6 2016, 3:26 PM
Gilles claimed this task.
Gilles added a subscriber: Gilles.

Resolved for the WMF with the global variable bumping. Unlikely to happen to 3rd-parties. If it does, the workaround is documented here.