Page MenuHomePhabricator

ResourceLoader: Don't add a version hash param in debug mode
Closed, ResolvedPublic

Description

As a workaround for T235667 and other possible cases where you might get a load.php URL while in debug mode.

In debug mode the cache control headers are set to no-cache, and many of the files served are direct paths to raw JS/CSS files. The same should apply to load.php URLs, as this would at least allow developers to set breakpoints without them being lost every time the hash changes (in the absence of source maps).

Event Timeline

In debug mode the cache control headers are set to no-cache

I'm not 100% sure that they are, but if they're not, they should be. And in any case, dropping the version param will lower the cache time to 5 minutes.

// See RFC 2616 § 14.19 ETag
// https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.19
header( 'ETag: ' . $etag );
if ( $context->getDebug() ) {
	// Do not cache debug responses
	header( 'Cache-Control: private, no-cache, must-revalidate' );
	header( 'Pragma: no-cache' );
} else {
	header( "Cache-Control: public, max-age=$maxage, s-maxage=$smaxage" );
	$exp = min( $maxage, $smaxage );
	header( 'Expires: ' . wfTimestamp( TS_RFC2822, $exp + time() ) );
}
Esanders added a subscriber: Krinkle.

a version hash that changes the URL every time the file is modified

We could work around this by dropping the version param whenever packageFiles and debug mode are used. Raw files in debug mode don't get a version param, so this would be making the behaviour match that.

Raw files in debug mode don't get a version param [..]

Can you elaborate? I agree we should make breakpoints easy and sticky, but as I understand it, urls change with classic file modules and debug mode as well. E.g. https://en.wikipedia.org/wiki/Main_Page?debug=true loads https://en.wikipedia.org/w/extensions/Echo/modules/api/mw.echo.api.NetworkHandler.js?6dc51, etc.

Hmm, not for me:

image.png (374×357 px, 65 KB)

Interesting, that seems like a bug. Possibly configuration related.

I don't have a preference for whether these urls have version parameters. However, note that their current presence in debug mode is not due to some generic code applying there. Rather, it was intentionally added to solve the problem outlined in T90983: ResourceLoader debug urls should bypass cache when they change. If we remove these, we'll have to re-solve that in a different way.

In a nut shell:

  • Static files from wgResourceBasePath are expected to have long-caching headers set at the web server level (depending on complexity of a dev setup, this is entirely optional, but it is something we do need to support in prod, in beta and when testing with MW-Vagrant with the Varnish module enabled).
  • In production we have multiple versions deployed at once from a single docroot, which means we need to disambiguate which version the current URL is for in a way that doesn't vary by hostname (all /w/resources and /static urls are cached in Varnish with host-agnostic cache key) . This applies mainly to font files and SVG icons.
  • Given raw debug files are served from the same directory, we need to cache-bust them, as well as to disambiguate. Using a content address/ hash param, solved both.

So it appears I am getting query strings for files in core, but not extensions. This might be because I have a non-standard directory structure (/extensions is a sibling folder of /core).

So I've been spared this annoying behaviour on all my files, due to a bug it seems.

That said adding breakpoints to files and expecting them to still be there when I make a small change seems like a pretty common workflow for devs.

It seems like removing the query string param, possibly using a separate config if required, would be a much simpler fix than implementing source maps.

@Esanders Can you check whether it works in latest Chrome when setting a breakpoint in a raw core file (with its version hash), and whether it remains after changing its content? I suspect this might work actually, and that the issue is with packageFiles having "load.php" as its filename which is ambiguous in this context (vs raw files having each their own url where the parameter can be ignored). In particular because a similar issue mentioned at T90983#2083366 was fixed upstream in Chromium.

Removing version hashes from load.php-debug responses seems fine by me. Would require a small change in the JS client to omit that param. The headers are already set to no-cache. Doing it for "raw" files is difficult given we don't control the response headers (per my previous comment).

Krinkle triaged this task as Medium priority.Oct 24 2019, 2:39 PM
Krinkle changed the task status from Open to Stalled.Feb 17 2020, 7:57 PM
Krinkle removed Krinkle as the assignee of this task.
Krinkle moved this task from Confirmed Problem to Backlog on the MediaWiki-ResourceLoader board.
Krinkle changed the task status from Stalled to Open.Aug 28 2021, 1:53 AM
Krinkle claimed this task.

The previous comments are about the static file requests, which is where most source code is served for VE under debug mode, and those are not subject to no-cache headers (since we're not able to influence the server on those static files served directly), and hence why we have version params on them for now. But, browsers handle those file afaik and I was not able to reproduce an issue there.

The task description, however, isn't about that, and I'm not sure why I went on this tangent. For source code served from load.php in debug mode (e.g. for package files), there is indeed no reason to add a version parameter there since we hav strong no-cache headers on those.

Change 715293 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] resourceloader: Make getVersionHash() final

https://gerrit.wikimedia.org/r/715293

Change 715295 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] resourceloader: Skip version hash calculation in debug mode

https://gerrit.wikimedia.org/r/715295

Change 715293 merged by jenkins-bot:

[mediawiki/core@master] resourceloader: Make getVersionHash() final

https://gerrit.wikimedia.org/r/715293

Change 715295 merged by jenkins-bot:

[mediawiki/core@master] resourceloader: Skip version hash calculation in debug mode

https://gerrit.wikimedia.org/r/715295

Change 723621 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] resourceloader: Add test for getVersionHash() in debug mode

https://gerrit.wikimedia.org/r/723621

Change 723621 merged by jenkins-bot:

[mediawiki/core@master] resourceloader: Add test for getVersionHash() in debug mode

https://gerrit.wikimedia.org/r/723621

Keeping open for the final step of removing them from debug=1 file references as well. While these seem to work in Chrome, they don't in Firefox.

Removing them is trivial for localhost development since there is no CDN, and so at most you'd need a hard refresh in the browser for any implicit/unofficial caching the browser is doing when there is no instruction for or against it. (Chrome does an in-memory cache here for a few minutes in those cases, similar to what it does with upload.wikimedia.org files in production.)

But for debugging in production we do have a CDN and we need some way of ensuring these won't stay in an outdated fashion for many hours/days after a deployment. I think we can provide some kind of affordance for this in WMF's /w/static.php but I need to think about it a bit more. This loosely relates to T285232#7377397, where there's also a potential change to this proxy that we might be able to accomodate in the same manner.

Closing as per previous comment. This is resolved in debug=2. And we're adding source maps in T47514 for more precise breakpoint setting. I'd rather not change debug=1 at this point as it's important that it remains a safe fallback for anything people might encounter with debug=2.