@ori is investigating, but so far we know it's not related to SyntaxHighlight. The page had an unclosed <math> tag followed by a lot of code and apparently something couldn't handle it.

HHVM 3.12.1 fails when a command-line argument to a process spawned using a LightProcess exceeds MAX_ARG_STRLEN, as defined in binfmts.h. MAX_ARG_STRLEN is the maximum length of strings passed to execve(). MAX_ARG_STRLEN is 131072 (or 128 kB).

We hit the limit because: (1) we parse JavaScript pages; (2) Gadget-popups.js contained a <math> tag in a comment, followed by 216 kB of JavaScript code; (3) the math extension tried to invoke texvc with that text as a command-line argument.

On HHVM 3.12.1, when using LightProcesses, this causes a crash.

Action items:

• add a check to wfShellExec that arguments are under MAX_ARG_STRLEN
• file a bug for HHVM indicating that it should check that arguments are < MAX_ARG_STRLEN before calling execve()
• add a sanity check to the math extension that skips shelling out to texvc if the input is insanely large

22:03 csteipp: deployed patch for T129506 to wmf15 & 16

@ori, are you planning on adding the sanity check to the Extension:Math to skip shelling out to texvc? If not, I'll take care of that.

In T129506#2124476, @dpatrick wrote:

@ori, are you planning on adding the sanity check to the Extension:Math to skip shelling out to texvc? If not, I'll take care of that.

I'd appreciate that.

Here are some revised patches, based on conversation with @csteipp.

The first patch revises Ori's patch above to use a constant:

This second patch uses the constant to skip shelling out to texvc:

Revised again. Moved constant definition to Defines.php and corrected some error message content:

In T129506#2219612, @dpatrick wrote:

Revised again. Moved constant definition to Defines.php and corrected some error message content:

T129506_00_MediaWiki.patch1 KBDownload

T129506_01_Math.patch1 KBDownload

This would make math ext depend on a specific version of mediawiki - thats probably fine, but we should check with ext maintainer to make sure (different exts seem to have different policies in regards to that sort of thing)

Its probably better to have a specific new error (input too long) then to reuse unknown error.

The math extension is automatically tagged with release and wmf tags. As long as the corresponding versions of math and core work with each other that's fine.
Since the master branch depends on core, it would be nice if the core patch could be applied before the math patch. Otherwise, Jenkins might get confused.

Advertisement:
Support the new math rendering mode that does not use the shell
I would very much like to get rid of the usage of shell command at all. cf. T74240 and T78046
The new rendering mode is now on beta.
The only thing that blocks it from becoming default are performance consideration for page editing. There is a pending config change by @mobrovac
https://gerrit.wikimedia.org/r/#/c/283269/
End of advertisement

In T129506#2222659, @Bawolff wrote:

In T129506#2219612, @dpatrick wrote:

Its probably better to have a specific new error (input too long) then to reuse unknown error.

Can we do that in a public change through gerrit? I'd like to move forward with getting the patches generated for the three other releases (and for both Mediawiki and Math).

Yeah, that sounds good

Patches for all releases:

Redeployed core patch (with define), and dependent Math patch.

18:17 csteipp: deployed revert/new patches for core & extension for T129506

Change 333309 had a related patch set uploaded (by Chad):
Skip shell invocation on large input

https://gerrit.wikimedia.org/r/333309

Change 333310 had a related patch set uploaded (by Chad):
Skip shell invocation on large input

https://gerrit.wikimedia.org/r/333310

Change 333311 had a related patch set uploaded (by Chad):
Skip shell invocation on large input

https://gerrit.wikimedia.org/r/333311

Change 333312 had a related patch set uploaded (by Chad):
Skip shell invocation on large input

https://gerrit.wikimedia.org/r/333312

Change 333312 merged by Chad:
Skip shell invocation on large input

https://gerrit.wikimedia.org/r/333312

Change 333309 merged by jenkins-bot:
Skip shell invocation on large input

https://gerrit.wikimedia.org/r/333309

Change 333311 merged by jenkins-bot:
Skip shell invocation on large input

https://gerrit.wikimedia.org/r/333311

Change 333310 merged by jenkins-bot:
Skip shell invocation on large input

https://gerrit.wikimedia.org/r/333310