Page MenuHomePhabricator

Many calls to syntaxhighlight is very slow, especially noticeable when previewing
Open, Needs TriagePublic

Description

To reproduce this error, create a page in the standard editor with 200 copies of
<source lang="xml" enclose="none"><Test></source>

and click "Preview"

The preview is never rendered, it just displays "Loading..."

However, with 199 copies it renders, although it takes a while - over 60 seconds.

Since this a round number, I suspect it is a threshold parameter. Given the length of time required for the preview it is probably not worth it to make it adjustable. But having the Preview just hangup is not a good solution. It would be better if the preview terminated with an error message rather than running indefinitely.

Note that it is still possible to save the edit, and the page itself renders fine. We are working around this by editing only by section, where the number of syntax highlight calls per section is less than the threshold.

Event Timeline

Taraathan raised the priority of this task from to Needs Triage.
Taraathan updated the task description. (Show Details)
Taraathan subscribed.

We are now experiencing a much more serious problem. We have a page with 811 distinct usages of syntax highlighting, mostly as inline fragments. It will not render - instead we get a timeout.

"we" ?

To me this sounds like you are missing some of the caching layer optimizations, rather than a problem with this specific extension.

"We" is the organization I work for.

Indeed, our wiki is on a shared host and there are many things we cannot do. The workaround we chose is to only use syntax highlighting for block quotes, and to go back to &lt;code>&lt;nowiki> tags for inline (XML) fragments.

But how do you explain the difference between 199 and 200 usages (slow rendering vs. hangup and eventual timeout)?

Just wondering - did you try what I suggested - making a page with 200 copies of <source lang="xml" enclose="none"><Test></source>

Did it work for you?

Tested a bit. It just takes that long to generate the output. The request times out before the server is able to finish it. On reading this isn't too much of a problem, since the server caches that, but a preview is the first render of something, so here your wait time is simply larger than you browser request time.

This also used to happen a lot with complex templates in the past, and here particularly it is caused by the fact that for each individual syntax highlight block, the renderer needs to start up a separate python process to generate the results. It's not a bug, but expected at this moment. But not really ideal either. I'll reword the description of the ticket.

TheDJ renamed this task from Edit Preview with too many calls to syntaxhighlight never completes to Many calls to syntaxhighlight is very slow, especially noticeable when previewing.Feb 24 2016, 7:35 PM

This problem still exist for the latest LTS of MW/extension. I wrote a comment https://www.mediawiki.org/wiki/Topic:Vw4yfp0viknw4ree before i found this ticket. As already mentioned the reason for the slow rendering is that for each code block a separate python process is needed. I wonder if the code could be rewritten in a way that only a single conversion could be performed by merging the input of all blocks before calling the converting only once.

There is a workaround we described in https://github.com/Khan/pygments-server/issues/1. It uses a HTTP server to handle the requests.