Page MenuHomePhabricator

LinkCache should be LRU based to avoid indefinitely growing and causing memory issues
Closed, ResolvedPublic

Description

This has been hacked around in the past (e.g. rECIR3d5a0e7479bb: Leak less memory in forceSearchIndex.php), but we should just make LinkCache lru-based to avoid using up lots of memory in long running maint scripts.

Event Timeline

Legoktm created this task.Jul 26 2015, 10:47 PM
Legoktm updated the task description. (Show Details)
Legoktm raised the priority of this task from to Needs Triage.
Legoktm added a project: MediaWiki-Cache.
Legoktm added subscribers: Legoktm, hoo, Reedy.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 26 2015, 10:47 PM
Reedy set Security to None.
MZMcBride added a subscriber: tstarling.

Change 227168 had a related patch set uploaded (by Legoktm):
Don't let LinkCache grow indefinitely

https://gerrit.wikimedia.org/r/227168

A single page can easily produce tens of thousands links.

Change 227168 merged by jenkins-bot:
Don't let LinkCache grow indefinitely

https://gerrit.wikimedia.org/r/227168

Legoktm closed this task as Resolved.Jul 27 2015, 5:58 PM

A single page can easily produce tens of thousands links.

I made LinkCache LRU-based, so if a page is linking to thousands of different pages, the LinkCache wouldn't have helped anyways. If some of the pages are the same, the LRU part should kick in and use the cache (unless we hit 10k unknown pages, then see the same link again).