Tagging monitoring as well because we generally associate HTTP 500 with application errors, and HTTP 503 with traffic/infra problems. If the PHP 7 roll out changes that, this would complicate some matters and make incident investigation more difficult.

Please provide the full responses, including headers, returned by the HHVM and PHP7 origin servers.

Hi, I've tested a few combinations of errors, and the the only case where this happens is when you choose action=segfault.

In that case, the 503 is returned directly from apache, so probably from php-fpm. If this behaviour changed from earlier, I guess the problem is in the code, as no configuration was changed.

Varnish doesn't do any transformation here and just reproduces what it's getting from the backend.

I changed the title of the task to reflect myt findings, and changed the associated tags accordingly

As I explained in T187147#5295715, my understanding is that in case of a segfault php-fpm fails to properly respond in any ways, forcing apache to produce a 503 error given it's working in proxy mode.

Using a modified version of furl that now supports unix sockets, for segfaults I get:

$ sudo furl --server unix:///run/php/fpm-www.sock --script /w/fatal-error.php --docroot /srv/mediawiki/docroot/wikipedia.org/ 'http://en.wikipedia.org/w/fatal-error.php?password=<redacted>&action=segfault'
Fatal error: Uncaught exception 'Adoy\FastCGI\ForbiddenException' with message 'Not in white list. Check listen.allowed_clients.' in /home/oblivian/furl:585
Stack trace:
#0 /home/oblivian/furl(450): Adoy\FastCGI\Client->wait_for_response()
#1 /home/oblivian/furl(648): Adoy\FastCGI\Client->request()
#2 /home/oblivian/furl(695): Adoy\FastCGI\doFcgiRequest()
#3 {main}

Looking at furl's code, this happens when the server closes the connection abruptly without sending back any FCGI data (EOF is reached and no data is left to return and no valid FCGI response has been sent).

There isn't much that can be done there from what I can see.