Page MenuHomePhabricator
Feed Advanced Search

Nov 4 2022

JFG claimed T322363: Data dumps for November aborted.

Many thanks for your prompt response. Yes the November dumps are available now.

Nov 4 2022, 5:17 PM · Dumps-Generation

Nov 3 2022

JFG created T322363: Data dumps for November aborted.
Nov 3 2022, 6:45 PM · Dumps-Generation

Dec 20 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Confirmed: still leaking without Content Translation.

Dec 20 2018, 4:01 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

@Jdlrobson: I still see massive leaking into gigabytes for idle background pages while I edit heavily on foreground pages.

Dec 20 2018, 9:38 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Dec 7 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

@pmiazga: Your work on phab:T208634 looks promising. Let me know whenever there's a potential fix to test.

Dec 7 2018, 1:04 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Nov 26 2018

JFG added a comment to T208634: Popups: [Spike, 16hrs] Replicate Popups memory leak without other extensions.

Any news? Can I help in any way?

Nov 26 2018, 8:59 PM · Web-Team-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2), Spike, Performance-Team (Radar), Page-Previews

Nov 3 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Sure. But please note the issue may not be related to popups.

Nov 3 2018, 9:21 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Oct 26 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

@kostajh, @Krinkle: bumping thread.

Oct 26 2018, 9:17 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Oct 18 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

@kostajh: The JavaScript memory is stable in the background page.

Oct 18 2018, 9:54 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Oct 17 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

@Krinkle: I tried the process you describe on the German Wikipedia, to avoid being polluted by my daily work on the English site. Here's what I have done:

  • disabled page preview in preferences
  • fully purged the browser cache
  • loaded a dewiki article in a new tab, making sure it had its own process ID (background page)
  • opened another dewiki article in another tab, making sure it had a separate process ID (foreground page)
  • browsed around in the foreground page, while checking memory usage on both pages
  • noticed memory usage of the background page increasing when left untouched, while browsing in the foreground page
  • saw an 8MB leap of memory usage on the background page when I hovered over a link in the foreground page. The link hovering just displayed a tooltip with the name of the target article, "Wirtschaftswunder".
  • saw a 12 MB leap on the background page when I opened another article in the foreground page, while it was being rendered
  • every new page rendered in the foreground process triggered a 7-8 MB memory increase in the background page's process
  • noticed some tiny amount of CPU usage in the background process every time I hovered over a link in the foreground page (about 0.1% CPU visible during one or two frames of the 1-second refresh rate of the task manager). That usually did not trigger a memory increase.
  • noticed some more significant CPU usage in the background process every time the foreground process finishes loading and rendering a new page (typically 2.7–3.5% during one frame), happening in sync with the memory increase of 7-8 MB. This usually happens with a delay of a couple seconds after the new page is loaded.
  • opening a picture within an article of the foreground process also triggers a slightly-delayed 7-8 MB memory increase and 1-second 3% CPU usage in the background process
  • browsing through several images in the foreground page once in gallery mode does not change the memory allocation in the background process, even on articles with many pictures.
  • throughout all these actions, memory usage of the active foreground process goes up and down, and remains far under what happens to the background process.
  • during all the activities in the foreground process, I have not seen any network usage triggered in the background process
  • when switching to the background page, memory usage remains high, it can even increase. Scrolling does not change it.
  • hovering over links in the background page (now foreground) does not reduce the memory footprint (that differs from the case when popups were active -- the popup display was clearing the extra memory), and it does consume some CPU (5-6% during 2-3 seconds). Again, only the tooltip with target article name gets displayed.
  • after clicking a link to load another article, memory is purged and returns to a normal level.
Oct 17 2018, 8:57 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

New test: the leak-in-background-page phenomenon also occurs when logged out.

Oct 17 2018, 1:41 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Another test whereby the leak is easily triggered with minimal steps:

  1. open a first page, read it, scroll around, let it come to a stable memory footprint (as monitored in the Chrome task manager)
  2. open another page in a different tab, make sure it has a different process ID, just let it render without moving the mouse into it
  3. watch the memory footprint of both pages and do a single mouseover on a link within the foreground page
  4. the background page consumes a few extra megabytes of memory!
  5. keep reading the foreground page; very soon the background page consumes more memory than the foreground page…
Oct 17 2018, 1:33 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

I just tested the working-in-foreground-while-idle-page-in-background case on the German Wikipedia. Confirmed that:

  1. the background page does not leak as long as it's the only open page on dewiki, it can be stable for hours
  2. even a small amount of work on another dewiki page (open article, click edit, click preview) triggers 30-40 MB of leak in the background page
  3. bringing the background page back to the foreground by itself causes no change, scrolling carefully is also fine
  4. triggering a popup with a mouseover on any link resets the memory usage to its baseline value from step 1

Please disregard my previous comments on the phenomenon being limited to enwiki: that was because I had not tested background pages on other wikis while working on foreground pages of the same wiki.

Oct 17 2018, 1:19 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Correction, and maybe an important clue: when a page in the background has been leaking memory, merely bringing it to the foreground and scrolling around in it is sufficient to reduce the memory footprint back to normal. No need to edit the page, just view it. With more careful experimenting, it appears that the memory is released as soon as a link mouseover event is processed and a thumbnail preview displayed. When I just open the relevant tab and carefully scroll from the sides without mousing over any text, the memory usage remains high. This behavior apparently establishes a strong connection between the popups system and the memory leak, and it's easy to reproduce. I tried to take a sample execution trace from the console to capture the release of memory, but that did not work because the leaked memory also gets released as soon as I open the dev tools panel. Hope this helps. Let me know what I can attempt next.

Oct 17 2018, 1:10 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Thanks for the new summary. I'll add that we have eliminated a bunch of other possible causes, by removing browser extensions, user scripts and gadgets. I started noticing those leaks in early September, as they became monumental (idle pages consuming more than 1 GB of memory before I kill them manually). I thought they might be related to the "new page reviewer" user right that I had recently acquired, but I have since opted out of this right and the problem persists.

Oct 17 2018, 10:55 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Oct 7 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Update: I have disabled the new page reviewer user permission, and the memory still leaks. Let me know if you need any new reports.

Oct 7 2018, 2:09 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Oct 1 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

@Krinkle: anything I can do to help track down the issue?

Oct 1 2018, 10:24 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Sep 25 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

May I ask how you found the memory increase originally? What tool or method did you use to find that it came from Wikipedia?

I have been watching memory usage in the Chrome task manager.

Sep 25 2018, 7:48 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Latest experiment: when I check "Disable access keys" in preferences, the ext.cite.a11y module is not loaded (neither are its friends jquery.tablesorter and jquery.makeCollapsible), but the page still leaks.

Sep 25 2018, 12:21 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Also, I don't think page previews are very relevant to the central case. I may have triggered that code by mousing over the page in this particular session, but most of my tests showing large long-term leaks happened with the window minimized, thus without any user interaction at all.

Sep 25 2018, 6:43 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Thanks for your detective work, @Krinkle. FYI I have taken longer samples but I could not see much relevant activity in there. Given the clear-cut difference in modules when a page leaks and does not leak, I would suggest to focus our investigations on the a11y service (accessibility). Does this belong to core mediawiki code? Is there a way to disable it in my account settings? Can I produce a memory trace that would specifically focus on events linked to this module?

Sep 25 2018, 6:41 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Sep 24 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

@Krinkle: Here are the requested object dumps, taken in safemode (because that loads less stuff, and still leaks):

  • Cookies:
Sep 24 2018, 6:17 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Confirmed: Kluivert leaks. (With all due respect to the footballers involved…)

Sep 24 2018, 5:47 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Sep 23 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

The simplest page I used when conducting long-term tests was https://en.wikipedia.org/wiki/DRG_Class_01. Just a few pictures and infoboxes in there, no tables or exotic templates. Trying on your "Kluivert" stub now, but reporting might take a few hours.

Sep 23 2018, 9:12 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Comparing the lists, here are the only modules loaded in a leaky situation (safemode) which are not loaded in the non-leaky case (safemode + incognito):

Sep 23 2018, 8:58 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

As a potentially helpful addition, here are the modules loaded in incognito mode (no leak):

Sep 23 2018, 8:31 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

@Krinkle: Thanks for your comments and suggestions. I have run a day-long experiment, so that I can confirm the following:

  • A page loaded in an incognito window while logged in does not leak.
  • A page loaded with ?safemode=1 while logged in leaks like a sieve.
  • All my Chrome extensions are disabled except AdBlock Plus. However AdBlock Plus is turned off on the Wikipedia domains.
Sep 23 2018, 8:28 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Sep 22 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

One thing I could try easily is logging out of Wikipedia, then if the issue disappears we will know that it must be linked to my enwiki user settings. Will do that now.

Sep 22 2018, 2:38 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Have you tried using another browser (Firefox probably) to see whether the problem is related to Google Chrome?

I have not. I've been using Wikipedia with Chrome for years, sometimes with hundreds of open tabs, and never seen this kind of behavior. Sure, it might be some phenomenon that only appears when you combine enwiki with Chrome, but that's a frequent enough combo among users to be taken seriously.

Sep 22 2018, 8:45 AM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Sep 21 2018

JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

I took a memory profiling excerpt from the JS console, but I can't upload it here. The system complains that "no configured storage engines can store this file." Now what?

Sep 21 2018, 8:03 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team
JFG added a comment to T205127: Memory consumption on Wikipedia pages running in background tabs grows as other pages are opened.

Thanks for reporting my issue here. Let me add that only the English Wikipedia suffers from this leak, which is why I suspected my user rights, scripts or gadgets there. To compare, I have kept pages open for several hours on the German, French, Japanese and Russian Wikipedias, as well as Wikidata, on the exact same browser and configuration, and all the non-English pages have remained stable.

Sep 21 2018, 7:54 PM · Web-Team-Backlog (Needs Prioritization (Tech)), Growth-Team-Filtering, Upstream, Page-Previews, Growth-Team

Jul 5 2018

JFG updated the task description for T198827: PDF URL not recognized as dead.
Jul 5 2018, 2:25 AM · InternetArchiveBot
JFG created T198827: PDF URL not recognized as dead.
Jul 5 2018, 2:23 AM · InternetArchiveBot

Jan 30 2018

JFG created T186012: Failure to mark dead URLs.
Jan 30 2018, 3:27 PM · InternetArchiveBot