CI: Get notified immediately when a job fails

If you've submitted patches for MediaWiki core, skins or extensions, you've seen this output in Gerrit:

image.png (498×1 px, 253 KB)

That is a list of links to each job's console output for a patch that failed verification.

You can see a job that failed at 1m 54s. But jenkins-bot does not post a comment on the patch until all jobs have completed. That means you won't get email/IRC notifications for test failures on your patch until the longest running job completes, in this case, after 14m 57s.[0] ⏳⏱️

With all due respect to xkcd/303... wouldn't it be nice to get notified as soon as a failure occurs, so you can fix your patch earlier to avoid context switching, or losing time during a backport window?

IMHO, yes, and, now it's possible!

⚙️ Get started

Commit a quibble.yaml file (documentation, example patch) to your MediaWiki project[1]:

    should_vote: 1
    should_comment: 1

The next time that there is a test failure[2] in your repository, you will see a comment from the Early warning bot and a Verified: -1 vote.

Here's an example of how that might look in practice:

image.png (1×1 px, 303 KB)
(Yes, the formatting needs some work still, patches welcome!)

So, the bot announces 2 minutes after the patch is updated that there's a problem, with the output of the failed command. The full report from jenkins-bot arrives 14 minutes later.

📚 Further reading

For details on how this works, please see the documentation for the Early warning bot. Your feedback and contributions are very welcome on T323750: Provide early feedback when a patch has job failures (feel free to tag T323750 with patches adding quibble.yaml to your project.)

🙌🏻 Thank you


[0] An alternative for getting real time progress is to watch Zuul TV

image.png (878×618 px, 118 KB)
. There is also the excellent work in T214068: Display Zuul status of jobs for a change on Gerrit UI but this does not generate email/IRC notifications or set a verification label.
[1] This will work for MediaWiki core, extensions, skins; in theory any CI job using Quibble could use it, though.
[2] Some jobs, like mwext-phan, won't report back early because they are not yet run via Quibble.

Written by kostajh on Mar 7 2023, 9:59 AM.
Staff Software Engineer, Trust & Safety Product
Urbanecm, hashar
"100" token, awarded by Lectrician1."Yellow Medal" token, awarded by Sgs."Yellow Medal" token, awarded by Tgr."Yellow Medal" token, awarded by thcipriani.

Event Timeline