We often see scaler processes (convert, avconv etc.) hang for various reasons. For example, we've seen convert and avconv both hang in some cases, when they can't allocate more memory than the one limited to them by MediaWiki's ulimit call.
We should both handle such cases and also contain them better than we do now, for security reasons.
Tim Starling and I have discussed various approaches for this in the past. I proposed an LD_PRELOAD wrapper that would abort whenever malloc() is unable to allocate memory (hackish, but might work). We could also use cgroups, as they're better at tracking resources than ulimits and they can also prove useful at containing what those processes can do in case of security exploits.
Finally, we can probably also use something like SIGALRM or non-blocking pipe calls so that the parent process counts the time of waiting for its child and, if times out, kills it.
Version: 1.21.x
Severity: enhancement