Page MenuHomePhabricator

libpcre-related performance opportunities
Open, Needs TriagePublic


Both HHVM and PHP7 use PCRE's JIT compiler to compile regular expressions into efficient byte code. Compilation is pretty expensive, and PCRE doesn't cache anything for you: it's up to the caller to decide whether (and how) to store and reuse the bytecode and other JIT data for a particular pattern. So while PHP7 and HHVM use the same library to execute patterns, each runtime has its own cache implementation.

As far as I know, we've never attempted to analyze the efficiency of the pattern cache and tune it for MediaWiki. There may be some low-hanging performance optimization opportunities there, since MediaWiki spends a lot of resources on regexp execution.

PCRE also allocates a block of 32K on the heap to use as a stack for its regexp virtual machine. The size of the stack is tunable, and in the past there was some discussion on the PHP internals list about increasing it, driven partly by this bug: The discussion died out, partly due to lack of real-world data. So that's another ripe target for instrumentation and tuning, in my opinion.

AFAIK there's no way to instrument this from PHP code. We'd either have to modify the runtime to collect and export PCRE JIT metrics. Alternately this could be done using 'perf' to trace PCRE function calls. On recent-ish Intel processors this can be done using LBR ( and the runtime overhead is low enough that you can sample in prod.

All of the above assumes the WMF doesn't disable the PCRE JIT. If it's disabled then obviously the first thing to look into is whether turning it on could be beneficial.

Event Timeline

ori created this task.Feb 21 2019, 5:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 21 2019, 5:14 PM
Joe added a comment.Feb 22 2019, 7:10 AM

So, while jit is enabled by default on PHP 7.2 (pcre.jit is 1 by default), I don't see how perf could help in knowing how full the JIT VM is (which is what we want to measure probably).

It should be possible to make a patch to the PHP code to get the internal value, or simply to make that value tunable by ourselves.

HHVM does caching of PCRE expressions, which for us was quite a problem given the amount of dynamic regexes we have (to the point Tim had to implement LRU eviction in their code).

According to php-fpm has a per-process cache that does FIFO eviction, which is probably worse-performing for us, given the amount of dynamic regexes we have to use.

It would indeed be interesting to measure the cache hit ratio in both HHVM and PHP 7 for our workload - I'm pretty sure the numbers are going to be bad, but again instrumenting this will need some perf sorcery - and time to read PHP's code.

A last note: we used Sury's prebuilt packages until now, but given we're going to rebuild them ourselves per T216712, we could easily patch php-fpm to report more data we're interested in.

@Krinkle I'm going to untag us, but let me know if there is CPT specific work needed.