Isolating a single request, I see that most of the time is spent in executing

v8::internal::VisitWeakList<v8::internal::JSFunction>

and that parsing, even when successful, requires ~ 180 seconds.

I am now trying to determine why the worker dies.

Strace gives little more information, besides the fact for each of these pages parsoid does hundreds of preprocessing requests to the MW API. Maybe some recursion limit is reached?

The request limit in Parsoid is set to 110s, after which the worker commits suicide.

I will blacklist this specific title in RESTBase for now.

The PHP parser also gives up with lots of errors like this on the page:

...
S08W039 Lua error: too many expensive function calls.
...

Anyway, looks like Parsoid is missing some resource limit / state to detect this scenario. But, bot-driven pages (which are basically proxies for a database table) are usually the ones that give Parsoid trouble.

In T155618#2949470, @mobrovac wrote:

I will blacklist this specific title in RESTBase for now.

Marco is it something that Ops could do in case of fire? If so, is the procedure written down somewhere?

@elukey apparently this needs a code deploy, which means accepting a pull request on github (sic) where not everyone from ops has the ability to merge a PR (I do as I'm an admin of the wikimedia github org, but YMMV), then you need to check that into the gerrit-based deploy repo, then restbase uses some ansible recipe (sic, again) to be deployed instead of scap3 or trebuchet.

So, we definitely need someone from the services team doing it.

The correct way to handle this would be, of course, to allow ops to control at least part of the blacklist via puppet, or to standardize the deployment process at least.

I remember there were talks about moving restbase to scap3, but I don't think that has happened until now.