Page MenuHomePhabricator

Don't use PHP files on-disk for lesscache
Closed, ResolvedPublic

Description

In order to run MediaWiki in the authoritative repo mode I need to tell HHVM about all PHP files that might possibly get required or included.

Currently I am just adding *.lesscache all files that happen to be in my $wgCacheDirectory but it would be better to have something more robust for automated production deployment.

Something like a maintenance script to do that would be great.

Event Timeline

saper raised the priority of this task from to Needs Triage.
saper updated the task description. (Show Details)
saper subscribed.

For a scalable and performant production environment one shouldn't use file caching for LESS to begin with.

It seems we currently do have them in production (e.g. mw1018:/tmp/mw-cache-1.27.0-wmf.9/lessphp_*.lesscache).

The new parsing library we use for Less has a better way to cache the intermediate representation. It defaults to .php files that are evaluated from disk, but it also supports a callback that is given serialised PHP data. We should consider using that instead.

In addition to this, we also have our own LESS caching layer. So it might even make sense to disable the internal one entirely.

Relevant code:

Options:

  • Disable Less_Parser's file cache and rely only on our own.
  • Configure Less_Parser to use a callback instead of file cache, and hook it up into one of our object caches.
  • Disable our own cache and configure Less_Parser to use our object caches.

The outcome will depend on how good Less_Parser is at re-using cache and evaluating whether the cache is stale or not.

Krinkle renamed this task from Need to be able to pre-generate cached PHP files (e.g. lesscache) for HHVM to Don't use PHP files on-disk for lesscache.Dec 18 2015, 9:38 PM
Krinkle set Security to None.
Krinkle added a project: Performance-Team.
Krinkle added a subscriber: ori.

if we go with serialized PHP, will HHVM Authoritative Repo mode be able to load it?

It seems these files show relatively high in XHGui's Memory Hog detection.

Here's a request for modules=mmv instrumented, https://performance.wikimedia.org/xhgui/run/view?id=56c9d64f7ed6cca41b13ad3e

Top 5 memory hogs:

run_init::/tmp/mw-cache-1.27.0-wmf.13/lessphp_qmxbqxz5374sko4ockwsck8ksowwk4k.lesscache12,744,904 bytes
run_init::/tmp/mw-cache-1.27.0-wmf.13/lessphp_a6c7hcdc0tckc4kswgs008gogoosgkk.lesscache12,682,000 bytes
Less_Parser::ArgString@11,793,856 bytes
run_init::/tmp/mw-cache-1.27.0-wmf.13/lessphp_qf2nq1iptf4o4co8ggcww04owk0oos8.lesscache831,224 bytes
APCBagOStuff::set595,576 bytes
JavaScriptMinifier::minify460,921 bytes

Change 303078 had a related patch set uploaded (by Krinkle):
resourceloader: Disable filesystem cache for Less

https://gerrit.wikimedia.org/r/303078

Change 303078 merged by jenkins-bot:
resourceloader: Disable filesystem cache for Less

https://gerrit.wikimedia.org/r/303078

Does this mean that after updating to this patch those files can be deleted?

Does this mean that after updating to this patch those files can be deleted?

Yes.