Page MenuHomePhabricator

Lua memory profiling support
Open, Needs TriagePublic

Description

The English Wiktionary is repeatedly running into memory problems, causing template expansion to fail with "Lua error: not enough memory".

We have identified potential memory hogs and have optimised the underlying Lua data structures to save space. However many of these efforts are based on trial&error, it does not seem to be possible to get memory allocation information on a Module / Template level.

The wgPageParseReport/limitReport included in the HTML only contains the used/max amount:

"limitreport-memusage": {"value":52428798, "limit":52428800}

That's not very helpful. Optimising the memory usage is difficult if it can't be measured in detail.

Is there anything you can suggest to help with memory profiling?

Event Timeline

jberkel created this task.Feb 28 2018, 10:58 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 28 2018, 10:58 AM
jberkel updated the task description. (Show Details)Feb 28 2018, 11:01 AM

removing Wiktionary as this seems to be about a single Wiktionary site.

Could you provide some specific examples?

@Aklapper currently failing pages: wind, night, do

Recent Grease pit discussion: light and Lua error: not enough memory

Splitting the data module into a per-code format would, I completely agree, increase the overhead in terms of function calls, but since most pages contain very few languages, I suspect that on average it would reduce overall server resource consumption. Since it is very hard for us to profile the things we do on wiki, we will be mostly stuck guessing about these types of things.

(User:TheDaveRoss)

A relatively fast-loading, but typical, example of the resulting problem is at:

https://en.wiktionary.org/wiki/wind

A year has passed, and this bug hasn't even been triaged. The memory problems on the English Wiktionary are still there, and ugly workarounds are needed to get some pages to render without errors.

Is there anything else we could try in the meantime?

Reedy added a subscriber: Reedy.Mar 9 2019, 8:44 PM

A year has passed, and this bug hasn't even been triaged.

Many bugs get fixed without ever being triaged