Page MenuHomePhabricator

localStorage nearly or totally filled with routine operation
Closed, DuplicatePublic

Description

beta commons localStorage nearly full

seen on beta labs while logged in as Selenium_user
seen in Chrome/Chromium, not checked in Firefox (yet)

consistent behavior seen:
initial load of VisualEditor in beta enwiki sets localStorage to 3.5MB
initial load of UploadWizard in beta commons sets localStorage to 2.5MB

routine navigation to pages like

continues to fill localStorage until it maxes out.

inconsistent behavior:

I do not have a consistent repro to max out localStorage. at first I thought that repeatedly hitting shift-reload/cancel on a ?veaction=edit page did it, but on my second attempt I was unable to max out localStorage that way


Version: 1.23.0
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=56778

Attached:

Details

Reference
bz56728

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:16 AM
bzimport added a project: MediaWiki-JavaScript.
bzimport set Reference to bz56728.
bzimport added a subscriber: Unknown Object (MLST).

Created attachment 13722
beta commons localStorage after loading UploadWizard

Attached:

Comment on attachment 13721
beta commons localStorage nearly full

I meant beta enwiki for this attachment

brion added a comment.Nov 7 2013, 6:12 PM

Can we determine the limits of available localStorage in some consistent way?

(In reply to comment #3)

Can we determine the limits of available localStorage in some consistent way?

I believe it is per-browser. Empirically I found that my Chromium allows 5MB while my Chrome allows 10MB.

See http://arty.name/localstorage.html (warning: fills up your local storage upon loading the page and gives you a status message when done)

brion added a comment.Nov 7 2013, 6:18 PM

So apparently the only way to find the limit is to keep adding data until you fill it up. Awesome.

http://dev-test.nemikor.com/web-storage/support-test/

^ looks like a frequent limit is 5MB, which in some browsers means 2.5M *UTF-16 code points*. That's going to be pretty tight for the amount of code we want to store. :(

Well, the limit can be worked around…

"Introducing the HTML5 Hard Disk Filler™ API" http://feross.org/fill-disk/

:P

ori added a comment.Nov 7 2013, 6:34 PM

Several points:

  • Can you isolate the size of the module storage? You can check it by running mw.loader.inspect('store') in a JavaScript console. (See 'totalSize' column).
  • Pruning old modules from the cache is deferred until the page / DOM is quiescent. What happens if you just let the page load and hang out for 5-10 seconds?
  • Is this reproducible outside the beta cluster? My suspicion is that the mtimes of modules is being continuously changed because of the way it is set up. Haven't investigated yet, but the way to check would be to load the startup module and compare module versions across reloads.
ori added a comment.Nov 7 2013, 6:56 PM

(In reply to comment #7)

My suspicion is that the mtimes of modules is being continuously changed
because of the way [beta] is set up.

Doesn't appear to be the case. I hacked up a quick script to check: https://gist.github.com/atdt/7359920. No changes were merged to master while I tested, though; still interested to know what happens then.

(In reply to comment #8)

(In reply to comment #7)

My suspicion is that the mtimes of modules is being continuously changed
because of the way [beta] is set up.

Doesn't appear to be the case. I hacked up a quick script to check:
https://gist.github.com/atdt/7359920. No changes were merged to master
while
I tested, though; still interested to know what happens then.

And BTW, if you do find config differences between beta enwiki and prod, please let me know. I have other intermittent/unreproduceable performance problems with Chrome (not Firefox) there that are becoming quite frustrating, seen in particular in VE and UploadWizard. I worry that these problems exist in production but are so flaky that they are not reported.

ori added a comment.Nov 7 2013, 7:32 PM

(In reply to comment #9)

(In reply to comment #8)

(In reply to comment #7)

My suspicion is that the mtimes of modules is being continuously changed
because of the way [beta] is set up.

Doesn't appear to be the case. I hacked up a quick script to check:
https://gist.github.com/atdt/7359920. No changes were merged to master
while
I tested, though; still interested to know what happens then.

And BTW, if you do find config differences between beta enwiki and prod,
please
let me know. I have other intermittent/unreproduceable performance problems
with Chrome (not Firefox) there that are becoming quite frustrating, seen in
particular in VE and UploadWizard. I worry that these problems exist in
production but are so flaky that they are not reported.

modules are updating pretty frequently on beta. 20 mins' worth here: https://gist.github.com/atdt/7359920#file-rl-mon-log. I'm running it again now *with* timestamps.

matmarex set Security to None.
matmarex removed a subscriber: Unknown Object (MLST).