Fri, Dec 7
Mon, Nov 26
Any news? Can I help in any way?
Nov 3 2018
Sure. But please note the issue may not be related to popups.
Oct 26 2018
Oct 18 2018
Oct 17 2018
@Krinkle: I tried the process you describe on the German Wikipedia, to avoid being polluted by my daily work on the English site. Here's what I have done:
- disabled page preview in preferences
- fully purged the browser cache
- loaded a dewiki article in a new tab, making sure it had its own process ID (background page)
- opened another dewiki article in another tab, making sure it had a separate process ID (foreground page)
- browsed around in the foreground page, while checking memory usage on both pages
- noticed memory usage of the background page increasing when left untouched, while browsing in the foreground page
- saw an 8MB leap of memory usage on the background page when I hovered over a link in the foreground page. The link hovering just displayed a tooltip with the name of the target article, "Wirtschaftswunder".
- saw a 12 MB leap on the background page when I opened another article in the foreground page, while it was being rendered
- every new page rendered in the foreground process triggered a 7-8 MB memory increase in the background page's process
- noticed some tiny amount of CPU usage in the background process every time I hovered over a link in the foreground page (about 0.1% CPU visible during one or two frames of the 1-second refresh rate of the task manager). That usually did not trigger a memory increase.
- noticed some more significant CPU usage in the background process every time the foreground process finishes loading and rendering a new page (typically 2.7–3.5% during one frame), happening in sync with the memory increase of 7-8 MB. This usually happens with a delay of a couple seconds after the new page is loaded.
- opening a picture within an article of the foreground process also triggers a slightly-delayed 7-8 MB memory increase and 1-second 3% CPU usage in the background process
- browsing through several images in the foreground page once in gallery mode does not change the memory allocation in the background process, even on articles with many pictures.
- throughout all these actions, memory usage of the active foreground process goes up and down, and remains far under what happens to the background process.
- during all the activities in the foreground process, I have not seen any network usage triggered in the background process
- when switching to the background page, memory usage remains high, it can even increase. Scrolling does not change it.
- hovering over links in the background page (now foreground) does not reduce the memory footprint (that differs from the case when popups were active -- the popup display was clearing the extra memory), and it does consume some CPU (5-6% during 2-3 seconds). Again, only the tooltip with target article name gets displayed.
- after clicking a link to load another article, memory is purged and returns to a normal level.
New test: the leak-in-background-page phenomenon also occurs when logged out.
Another test whereby the leak is easily triggered with minimal steps:
- open a first page, read it, scroll around, let it come to a stable memory footprint (as monitored in the Chrome task manager)
- open another page in a different tab, make sure it has a different process ID, just let it render without moving the mouse into it
- watch the memory footprint of both pages and do a single mouseover on a link within the foreground page
- the background page consumes a few extra megabytes of memory!
- keep reading the foreground page; very soon the background page consumes more memory than the foreground page…
I just tested the working-in-foreground-while-idle-page-in-background case on the German Wikipedia. Confirmed that:
- the background page does not leak as long as it's the only open page on dewiki, it can be stable for hours
- even a small amount of work on another dewiki page (open article, click edit, click preview) triggers 30-40 MB of leak in the background page
- bringing the background page back to the foreground by itself causes no change, scrolling carefully is also fine
- triggering a popup with a mouseover on any link resets the memory usage to its baseline value from step 1
Please disregard my previous comments on the phenomenon being limited to enwiki: that was because I had not tested background pages on other wikis while working on foreground pages of the same wiki.
Correction, and maybe an important clue: when a page in the background has been leaking memory, merely bringing it to the foreground and scrolling around in it is sufficient to reduce the memory footprint back to normal. No need to edit the page, just view it. With more careful experimenting, it appears that the memory is released as soon as a link mouseover event is processed and a thumbnail preview displayed. When I just open the relevant tab and carefully scroll from the sides without mousing over any text, the memory usage remains high. This behavior apparently establishes a strong connection between the popups system and the memory leak, and it's easy to reproduce. I tried to take a sample execution trace from the console to capture the release of memory, but that did not work because the leaked memory also gets released as soon as I open the dev tools panel. Hope this helps. Let me know what I can attempt next.
Thanks for the new summary. I'll add that we have eliminated a bunch of other possible causes, by removing browser extensions, user scripts and gadgets. I started noticing those leaks in early September, as they became monumental (idle pages consuming more than 1 GB of memory before I kill them manually). I thought they might be related to the "new page reviewer" user right that I had recently acquired, but I have since opted out of this right and the problem persists.
Oct 7 2018
Update: I have disabled the new page reviewer user permission, and the memory still leaks. Let me know if you need any new reports.
Oct 1 2018
@Krinkle: anything I can do to help track down the issue?
Sep 25 2018
May I ask how you found the memory increase originally? What tool or method did you use to find that it came from Wikipedia?
I have been watching memory usage in the Chrome task manager.
Latest experiment: when I check "Disable access keys" in preferences, the ext.cite.a11y module is not loaded (neither are its friends jquery.tablesorter and jquery.makeCollapsible), but the page still leaks.
Also, I don't think page previews are very relevant to the central case. I may have triggered that code by mousing over the page in this particular session, but most of my tests showing large long-term leaks happened with the window minimized, thus without any user interaction at all.
Thanks for your detective work, @Krinkle. FYI I have taken longer samples but I could not see much relevant activity in there. Given the clear-cut difference in modules when a page leaks and does not leak, I would suggest to focus our investigations on the a11y service (accessibility). Does this belong to core mediawiki code? Is there a way to disable it in my account settings? Can I produce a memory trace that would specifically focus on events linked to this module?
Sep 24 2018
@Krinkle: Here are the requested object dumps, taken in safemode (because that loads less stuff, and still leaks):
Confirmed: Kluivert leaks. (With all due respect to the footballers involved…)
Sep 23 2018
The simplest page I used when conducting long-term tests was https://en.wikipedia.org/wiki/DRG_Class_01. Just a few pictures and infoboxes in there, no tables or exotic templates. Trying on your "Kluivert" stub now, but reporting might take a few hours.
Comparing the lists, here are the only modules loaded in a leaky situation (safemode) which are not loaded in the non-leaky case (safemode + incognito):
As a potentially helpful addition, here are the modules loaded in incognito mode (no leak):
@Krinkle: Thanks for your comments and suggestions. I have run a day-long experiment, so that I can confirm the following:
- A page loaded in an incognito window while logged in does not leak.
- A page loaded with ?safemode=1 while logged in leaks like a sieve.
- All my Chrome extensions are disabled except AdBlock Plus. However AdBlock Plus is turned off on the Wikipedia domains.
Sep 22 2018
One thing I could try easily is logging out of Wikipedia, then if the issue disappears we will know that it must be linked to my enwiki user settings. Will do that now.
Have you tried using another browser (Firefox probably) to see whether the problem is related to Google Chrome?
I have not. I've been using Wikipedia with Chrome for years, sometimes with hundreds of open tabs, and never seen this kind of behavior. Sure, it might be some phenomenon that only appears when you combine enwiki with Chrome, but that's a frequent enough combo among users to be taken seriously.
Sep 21 2018
I took a memory profiling excerpt from the JS console, but I can't upload it here. The system complains that "no configured storage engines can store this file." Now what?
Thanks for reporting my issue here. Let me add that only the English Wikipedia suffers from this leak, which is why I suspected my user rights, scripts or gadgets there. To compare, I have kept pages open for several hours on the German, French, Japanese and Russian Wikipedias, as well as Wikidata, on the exact same browser and configuration, and all the non-English pages have remained stable.