If you've submitted patches for MediaWiki core, skins or extensions, you've seen this output in Gerrit:

That is a list of links to each job's console output for a patch that failed verification.

You can see a job that failed at 1m 54s. But jenkins-bot does not post a comment on the patch until all jobs have completed. That means you won't get email/IRC notifications for test failures on your patch until the longest running job completes, in this case, after 14m 57s.[0] ⏳⏱️

With all due respect to xkcd/303... wouldn't it be nice to get notified as soon as a failure occurs, so you can fix your patch earlier to avoid context switching, or losing time during a backport window?

IMHO, yes, and, now it's possible!

⚙️ Get started

Commit a quibble.yaml file (documentation, example patch) to your MediaWiki project[1]:

earlywarning:
    should_vote: 1
    should_comment: 1

The next time that there is a test failure[2] in your repository, you will see a comment from the Early warning bot and a Verified: -1 vote.

Here's an example of how that might look in practice:

So, the bot announces 2 minutes after the patch is updated that there's a problem, with the output of the failed command. The full report from jenkins-bot arrives 14 minutes later.

📚 Further reading

For details on how this works, please see the documentation for the Early warning bot. Your feedback and contributions are very welcome on T323750: Provide early feedback when a patch has job failures (feel free to tag T323750 with patches adding quibble.yaml to your project.)

🙌🏻 Thank you

• @hashar for code & architecture review, pushing through Jenkins configuration updates, and solving some low-level plumbing for T331061: Capture output from failed command and transmit to earlywarningbot
• @Urbanecm for fixing a bug where the bot reported on non-essential pipelines (T331236)

Cheers,
Kosta

[0] An alternative for getting real time progress is to watch Zuul TV