Page MenuHomePhabricator
Create Task
Maniphest T106998

LinkCache should be LRU based to avoid indefinitely growing and causing memory issues
Closed, ResolvedPublic
Actions

Description

This has been hacked around in the past (e.g. rECIR3d5a0e7479bb: Leak less memory in forceSearchIndex.php), but we should just make LinkCache lru-based to avoid using up lots of memory in long running maint scripts.

Details

SubjectRepoBranchLines +/-
Don't let LinkCache grow indefinitelymediawiki/coremaster+59 -25
Customize query in gerrit

Related Objects

Event Timeline

Legoktm created this task.Jul 26 2015, 10:47 PM
Legoktm raised the priority of this task from to Needs Triage.
Legoktm updated the task description. (Show Details)
Legoktm added a project: MediaWiki-libs-BagOStuff.
Legoktm added subscribers: Legoktm, hoo, Reedy.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 26 2015, 10:47 PM
Reedy added a project: Performance Issue.Jul 26 2015, 10:48 PM
Reedy set Security to None.
Legoktm claimed this task.Jul 26 2015, 10:54 PM
Legoktm mentioned this in T104278: E7. Run populateContentModel.php for page on all WMF wikis.
• MZMcBride added a subscriber: tstarling.Jul 26 2015, 11:03 PM
• MZMcBride subscribed.
gerritbot subscribed.Jul 26 2015, 11:08 PM

Change 227168 had a related patch set uploaded (by Legoktm):
Don't let LinkCache grow indefinitely

https://gerrit.wikimedia.org/r/227168

gerritbot added a project: Patch-For-Review.Jul 26 2015, 11:08 PM
Nemo_bis subscribed.Jul 27 2015, 8:36 AM

A single page can easily produce tens of thousands links.

gerritbot added a comment.Jul 27 2015, 5:53 PM

Change 227168 merged by jenkins-bot:
Don't let LinkCache grow indefinitely

https://gerrit.wikimedia.org/r/227168

Legoktm mentioned this in rMW32d1017e7d77: Don't let LinkCache grow indefinitely.Jul 27 2015, 5:54 PM
Legoktm closed this task as Resolved.Jul 27 2015, 5:58 PM
In T106998#1484144, @Nemo_bis wrote:

A single page can easily produce tens of thousands links.

I made LinkCache LRU-based, so if a page is linking to thousands of different pages, the LinkCache wouldn't have helped anyways. If some of the pages are the same, the LRU part should kick in and use the cache (unless we hit 10k unknown pages, then see the same link again).

• Forrestbot added projects: MW-1.26-release, WMF-deploy-2015-07-28_(1.26wmf16).Jul 27 2015, 6:00 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits