Page MenuHomePhabricator

Re-enable ResourceLoader store in Firefox
Closed, ResolvedPublic

Description

In most browsers Resource Loader caches modules in LocalStorage in order to not rerequest the same modules on every page-load. However, this is not the case for Firefox, as Firefox used to apply the allowed storage quota per domain, not per origin. Thus, the use of LocalStorage was disabled for Firefox in T66721: mw.loader.store should not occupy all of localStorage.

However, with Firefox 69 this quota should now be per origin: https://bugzilla.mozilla.org/show_bug.cgi?id=1064466#c18. So it should be possible to reenable the use LocalStorage again for users with a current Firefox.

I'll provide a patch as basis for discussion.

Ref T121646.

Event Timeline

Change 544183 had a related patch set uploaded (by Michael Große; owner: Michael Große):
[mediawiki/core@master] Enable LocalStorage for Firefox >= 69

https://gerrit.wikimedia.org/r/544183

Even though the bug mentioned above indicates that the new LocalStorage should be available in Firefox 69, I would feel better if we were able to actually verify that.
However, I'm not sure how to do so. Any ideas?

Aklapper renamed this task from Enable LocalStorage for FireFox 69.0+ to Enable LocalStorage for Firefox 69.0+.Oct 19 2019, 12:54 PM

@Michael You could try something like the following:

localStorage.clear();
localStorage.setItem('mykey-a', '1234567890'.repeat(1e5));
localStorage.setItem('mykey-b', '1234567890'.repeat(1e5));
localStorage.setItem('mykey-c', '1234567890'.repeat(1e5));
// … etc.

from the console on https://www.wikipedia.org in Firefox 68 and see how far you can get before it throws with "Storage quota exceeded". I expect it to be full after around 6 or 7 such keys being stored. Then navigate to "English" / https://en.wikipedia.org and repeat the same.

If the storage quota is shared (like it used to) then you will not be able to store even 1 such key after localStorage.clear() on en.wikipedia.org, the same way it was denied from www.wikipedia.org.

On the newer Firefox versions, you will be able to store roughly the same number of keys on both independently. You could use something like https://browserstack.com to access the older browser versions (various open-source orgs have been given accounts there, don't know about WMDE).

Krinkle renamed this task from Enable LocalStorage for Firefox 69.0+ to Enable ResourceLoader module storage on Firefox 69+.Oct 19 2019, 7:34 PM
Krinkle assigned this task to Michael.
Krinkle moved this task from Inbox to Accepted Enhancement on the MediaWiki-ResourceLoader board.
Krinkle moved this task from Inbox, needs triage to Blocked (old) on the Performance-Team board.
Krinkle triaged this task as Medium priority.Oct 19 2019, 7:35 PM

If I understand https://bugzilla.mozilla.org/show_bug.cgi?id=1592136 correctly, this has been reverted temporarily in FF 70.0.1.

Yeah, I later noticed that this config option seemed to be only active by default in the 69 beta, not in the actual release and wanted to investigate further, but didn't get around to it so far.
At least now we have enough time to get the implementation right without "loosing time" to Wikipedia being less performative than it could be on Firefox.

Krinkle changed the task status from Open to Stalled.Dec 20 2019, 3:46 PM

Stalled per upstream Firefox.

Change 544183 abandoned by Krinkle:
Enable LocalStorage for Firefox >= 69

Reason:
Closing to clear review dash. Feel free re-open if/when upstream Firefox is fixed.

https://gerrit.wikimedia.org/r/544183

Krinkle moved this task from Backlog to Reported Upstream on the Upstream board.

Upstream Firefox recomend using the ServiceWorker Cache API instead of localStorage, which is currently not subject to this bug.

https://bugzilla.mozilla.org/show_bug.cgi?id=1064466#c37

In other news, upstream V8/Chromium also recommended the same thing when I asked about bytecode caching (tweet).

We actually have a task for that already, see T101732: Use Service Worker cache when available for ResourceLoader caching instead of LocalStorage.

Krinkle changed the task status from Stalled to Open.Jan 23 2022, 1:28 AM

Mozilla rolled out the next-gen LSNG feature in Firefox 92, which resolved the quota issue. We now have a per-origin quota of 5MB, the same as in other browsers. The abuse prevention mechanism at the eTLD+1 level remains, but it was raised to 10GB which avoids most practical implications.

Comment by Jan Varga from mozilla bug 1064466 comment #c40:
  • Quota checks and usage reporting for LS is shared with IDB and Cache API.
  • The logical size of LS data is tracked ([others] track the physical size of files on disk).
  • LS data is evicted along with IDB and Cache API.

There are a number of other interesting performance improvements that went along this, most of which are tracked under mozilla bug 1286798. Such as:

Jan Varga wrote at mozilla bug 1722668:

Some users have a lot of free space on disk so origins are never evicted which leads to large set of directories and files after some time (years). This can be a problem on slower hardware, so we should reduce number of origin directories if the initialization takes longer than X seconds. Origins which were used in last 13 months shouldn't be removed.

I verified it on Firefox 96 using the following snippet:

i = 0; while (i++ < 10000) { localStorage.setItem('filler' + i, (location.host + ' foo ').repeat(1024)); }
// Runs until: Uncaught DOMException: The quota has been exceeded.
  • On https://nl.wiktionary.org, open the console and run it. Then switch to the Storage tab in devtools and confirm there is data. Sort keys descending, and take note of the first key, which indicates how far it got before storage was full. In my case filler171.
  • On https://de.wiktionary.org, do the same. You should be able to get similarly far. If there was a shared quota, you'd get little to no space here.
  • On https://en.wiktionary.org, do the same.
  • Back on https://nl.wiktionary.org, reload the tab and confirm the data is still there.

Unassigning myself for now as part of the Bug management/Assignee cleanup. Might assign myself again to it in the future when I find time to work on it.

Change 776261 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] resourceloader: Enable mw.loader.store in Firefox

https://gerrit.wikimedia.org/r/776261

Krinkle renamed this task from Enable ResourceLoader module storage on Firefox 69+ to Re-enable ResourceLoader store in Firefox.Apr 1 2022, 9:48 PM
Krinkle moved this task from Reported Upstream to Patch merged upstream on the Upstream board.

Change 776261 merged by jenkins-bot:

[mediawiki/core@master] resourceloader: Enable mw.loader.store in Firefox

https://gerrit.wikimedia.org/r/776261

Keeping open to remind myself to verify and observe impact on roll-out.

Krinkle closed this task as Resolved.EditedApr 11 2022, 10:35 PM

From Wikipedia user journey in which four pages are visited (login, mainpage, obama, fb).

capture.jpg (1×2 px, 395 KB)

  1. Login. Cold, no expected and no actual difference (graph amplifies ±1KB of standard deviation). Per T145498, storage is not initialised here.
  2. Main page. Warm HTTP cache and cold module cache. Primes the module cache here. No expected or actual difference.
  3. Obama page. Warm module cache, much of the module payload is re-used from the defragmented cache primed on the main page. Reduction of 120KB from 220 down to 100KB.
  4. Fb page. Warm module cache, even more is re-used. Reduction of 150KB from 400 down to 250KB.

More details on Page drilldown:

Screenshot 2022-04-11 at 23.42.17.png (1×2 px, 522 KB)