This is happening again. Easy to reproduce at https://commons.wikimedia.org/wiki/Special:VipsTest?thumb=File%3Aexample.jpg&width=100&sharpen=0.8.

HTTP 500:

Error fetching URL: Failed to connect to 10.2.1.21 port 80: Connection refused

The IP in question comes from wmf-config:

$wgVipsThumbnailerHost = '10.2.1.21';

This IP, oddly, resolves to proton.svc.codfw.wmnet, which is unrelated (Proton is the Chromium PDF rendering service)

$ curl proton.svc.codfw.wmnet
curl: (7) Failed to connect to proton.svc.codfw.wmnet port 80: Connection refused

Looks like it originally pointed to the imagescaler pool, which was removed (T188062, 6d0b546710b70), and then the LVS IP was re-used for Proton (T186748 , dc7dff1db24ea62).

When the imagescaler pool was decom'ed, this reference in wmf-config should have been updated to point to the general app server pool instead, or disabled given we don't need to proxy anymore as each app server is now capable of doing this locally.

Change 446639 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Unset $wgVipsThumbnailerHost

https://gerrit.wikimedia.org/r/446639

Change 446639 merged by jenkins-bot:
[operations/mediawiki-config@master] Unset $wgVipsThumbnailerHost

https://gerrit.wikimedia.org/r/446639

Mentioned in SAL (#wikimedia-operations) [2018-07-18T19:24:19Z] <reedy@deploy1001> Synchronized wmf-config/CommonSettings.php: Unset wgVipsThumbnailHost T199937 (duration: 00m 55s)