thumb.php ignores the return status of MediaTransformOutput::streamFile() and/or FileRepo::streamFile() and returns a normal HTTP status even if the file could not be streamed, which results in Varnish caching the error. It should return 503 instead.
Per Timo's comment on the patchset, the streamFile calls in img_auth.php, SpecialUndelete#showFile, SpecialRevisiondelete#tryShowFile and SpecialUploadStash#outputLocalFile should be handled similarly. (Also, if we are going to touch all streamFile calls, it should probably be changed to return the Status object so the error messages can be more informative.) Less of a priority though, those can't get cached.
update: split to T92903
Just break MediaTransformOutput::streamFile() and/or FileRepo::streamFile() and make them return false. The first should affect newly created thumbnails (on a wiki using a 404 handler)- The second, I am not sure; I think it only happens on Swift (or similard shared-storage backends), but you can simulate it by calling thumb.php directly for an existing non-thumbnail file (/w/thumb.php?f=<title> should do it).