Page MenuHomePhabricator

Pygmentize times out on Windows
Open, Needs TriagePublic

Description

Running SyntaxHighlight on windows ends up with a timeout from Pygmentize, more or less as described here on MWwiki (the only difference is that I'm testing it with CACHE_NONE and still doesn't work). I found out this problem after T199789. Building the command via concatenation actually makes the process run, but the browser starts loading eternally without any error being returned.
I tried to run pygmentize via cmd directly, i.e.

path/to/pygmentize -l php -f html -O cssclass=mw-highlight,encoding=utf-8,startinline=1

Then added a sample string, like 1==1 via stdin, then Ctrl+Z to terminate input and the output is indeed correct (and returned instantly). Together with the fact that, at this point, the final command is the one I wrote above, this makes me think of some problem in Command::input; maybe it fails to properly terminate input sequence?

Also, I don't think it'll be really relevant but here are my specs: running XAMPP with PHP 7.2.2 on Windows 7, python 3.6.3.

Event Timeline

Daimona created this task.Jul 19 2018, 10:57 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 19 2018, 10:57 AM

I guess this isn't easy at all. I could experimentally determine what causes the script to hang: it all happens on this line, when reading the first pipe (should be stdin). I also read on the web that stream management is quite bugged on windows from PHP's end, although it's not clear whether those bugs are actually fixed.

Wellll yes, sounds like this is due to the many bugs of streams on windows reported on the web (just search for something like "php windows proc_open" or similar stuff to read a bunch of them). Trying to read from the stream before entering the while loop results in eternal loading for pipes[1] and pipes[2] while pipes[1] is readable, whatever function is used. Instead, if I change $desc to use files instead of pipes for stdout and stderr, everything works fine, with the output being sent to a file as expected. Sounds like a proper solution isn't easy to implement; maybe some workaround would be possible?

Daimona renamed this task from Pygmentize times out on Windows (due to failure of Command::input?) to Pygmentize times out on Windows.Jul 20 2018, 1:59 PM

I think I figured out the reason: stream_set_blocking doesn't work for pipes on windows, see doc and a linked bug report. I also printed the return value for these calls, and indeed they're all false. Apparently this won't be fixed on PHP's end since there actually isn't a good solution available on windows. I'm wondering whether there's a workaround that we can apply locally.