Following up from the declined T225871: Selenium and PHPUnit: Stop execution on failure and tangential to T248531: Abort a Zuul pipeline when one job completed with failures (change zuul scheduler's failure check from areAllJobsComplete to didAnyJobFail):
- have a web app on toolforge, i.e. ci-early-failure-notice.toolforge.org
- patch Quibble so that when a process ends with exit code other than 0, it POSTs some data about the job (patch details) to the web app
- the web app adds a comment and a Verified -1 to the patch
- the comment should include a link to the Jenkins build URL for the failing job
- [bonus points] Quibble could capture a formatted version of the failing command output, and include this in the comment that the web app leaves in Gerrit. That is related to T209149: Have linters/tests results show up as comments in files on gerrit. We'd need a trimmed version of this for PHPUnit which currently dumps a lot of text.
- developers receive pings early via email/IRC that a patch has failures. Yes, you can watch Zuul, but most people don't know to do that, and it doesn't notify you that a job has failed.
- Early feedback on a failed patch allows the developer to 1) fix the patch before switching their context to other things or 2) in a backport situation, know that they need to restart the jobs