Page MenuHomePhabricator

Can we cache the result of executeModule to improve #invoke startup time?
Open, Needs TriagePublic

Description

Most invokes spend the vast majority of their running time inside executeModule, even if the user function executed by the invoke is very small. T357199: Could we optimize Lua environment setup to improve {{#invoke:}} performance? improved the performance of executeModule by ~15%, but articles with thousands of invokes are still spending a significant portion of their parse time in executeModule.

When there are this many invokes, usually it's a fairly small number of modules getting used repeatedly. Could we possibly cache the LuaSandboxFunction chunks returned by executeModule, instead of recomputing them every time?

The most obvious way to do this would keep the module scope between invokes, which is undesirable (T67258: Information can be passed between #invoke's (tracking)). I have experimented wrapping the entire module in a closure, which solves the most obvious scope-sharing issues (and speeds up #invoke calls by about 5x!!!), but this still can leak globals. If that approach doesn't work, what about modifying lua_newstate's memory allocator to "reset" the scope to a previous known/safe value?

Event Timeline

pinging @tstarling as the person most likely to be able to answer if this is a viable approach and performance improvement.

Change #1228547 had a related patch set uploaded (by SomeRandomDeveloper; author: SomeRandomDeveloper):

[mediawiki/extensions/Scribunto@master] Allow using a shared global environment for module invocations

https://gerrit.wikimedia.org/r/1228547

@SomeRandomDeveloper As you are not subscribed to that other ticket, in T357199#10290324 I suggested another approach that would have a better chance of being accepted (as it preserves context isolation): lazy‑loading the mw.* libraries on first call rather than cloning everything from the beginning… However, I don’t know for sure whether it would be technically doable.

Change #1228547 merged by jenkins-bot:

[mediawiki/extensions/Scribunto@master] Allow using a shared global environment for module invocations

https://gerrit.wikimedia.org/r/1228547