Page MenuHomePhabricator

Cannot use SyntaxHighlight: "Unable to run external programs, proc_open() is disabled"
Closed, DeclinedPublicFeature


Hi Everyone,

I think this is a feature request. We run a [semi] hardened PHP installation according to 25 PHP Security Best Practices For Sys Admins. Many functions are removed, including proc_close, proc_get_status, proc_nice, proc_open, proc_terminate, and shell_exec. Also see our security.ini, and the list of disable_functions.

We are unable to run SyntaxHighlight due to its use of some of the disabled functions.

It would be nice if SyntaxHighlight were modified to allow its use in a hardened system.

Here is what happens when we try to enable SyntaxHighlight in our configuration.

# ./maintenance/update.php 

Fatal error: Uncaught ExtensionDependencyError: SyntaxHighlight requires "shell" ability: Unable to run external programs, proc_open() is disabled
 in /var/www/html/w/includes/registration/ExtensionRegistry.php:334
Stack trace:
#0 /var/www/html/w/includes/registration/ExtensionRegistry.php(186): ExtensionRegistry->readFromQueue(Array)
#1 /var/www/html/w/includes/Setup.php(143): ExtensionRegistry->loadFromQueue()
#2 /var/www/html/w/maintenance/doMaintenance.php(83): require_once('/var/www/html/w...')
#3 /var/www/html/w/maintenance/update.php(277): require_once('/var/www/html/w...')
#4 {main}
  thrown in /var/www/html/w/includes/registration/ExtensionRegistry.php on line 334

Event Timeline

Just because an article on the internet says it's a good idea, doesn't necessarily mean it is ;)

It should be noted that we did put some restrictions in place with T182468: Restrict pygments with firejail

Aklapper renamed this task from SyntaxHighlight uses dangerous functions to Cannot use SyntaxHighlight: "Unable to run external programs, proc_open() is disabled".Apr 21 2020, 6:47 AM
Aklapper changed the subtype of this task from "Bug Report" to "Feature Request".Apr 21 2020, 6:49 AM

I boldly renamed this task to summarize the actual problem, instead of having debatable statements such as "dangerous functions".

This is actually a regression when the extension moved from GeSHi (PHP) to Pygments (Python): T85794: Convert SyntaxHighlight_Geshi from Geshi to Pygments (was: Don't register 250+ modules) Since MW is PHP the extension can just use GeSHi as a library without having to fork a separate process via one of the unsafe functions removed in the hardened PHP.

To fix the problem we would have to move to a PHP-based syntax highlighter (or somehow embedded Python which is probably ugly and even more unsafe). We could move back to GeSHi, but I believe the move was done because GeSHi is basically dead with no development so we would have to fork it and maintain such ourselves. As another option, we could move to a different PHP syntax highlighter like:

I am not sure how this would impact a PHP hardened deployment of MediaWiki, but since WMF now uses Scribunto across the board, another option would be to move to some sort of Lua syntax highlighter like: This requires: LPeg is pattern-matching library for Lua, based on Parsing Expression Grammars (PEGs) by Roberto Ierusalimschy, one of the same people that created Lua. Adding that to Scribunto would be nice and would allow for a real regular expression pattern matching instead of the more restrictive one included with Lua.

Legoktm added a subscriber: Legoktm.

For the forseeable future, SyntaxHighlight will require some form of shelling out to Pygments. Work on using a microservice over HTTP is tracked as T257988: Optional use of and discussed in other tasks.