If you've submitted patches for MediaWiki core, skins or extensions, you've seen this output in Gerrit:
That is a list of links to each job's console output for a patch that failed verification.
You can see a job that failed at 1m 54s. But jenkins-bot does not post a comment on the patch until all jobs have completed. That means you won't get email/IRC notifications for test failures on your patch until the longest running job completes, in this case, after 14m 57s.[0] ⏳⏱️
With all due respect to xkcd/303... wouldn't it be nice to get notified as soon as a failure occurs, so you can fix your patch earlier to avoid context switching, or losing time during a backport window?
IMHO, yes, and, now it's possible!
⚙️ Get started
Commit a quibble.yaml file (documentation, example patch) to your MediaWiki project[1]:
earlywarning: should_vote: 1 should_comment: 1
The next time that there is a test failure[2] in your repository, you will see a comment from the Early warning bot and a Verified: -1 vote.
Here's an example of how that might look in practice:
[3](Yes, the formatting needs some work still, patches welcome!)
So, the bot announces 2 minutes after the patch is updated that there's a problem, with the output of the failed command. The full report from jenkins-bot arrives 14 minutes later.
📚 Further reading
For details on how this works, please see the documentation for the Early warning bot. Your feedback and contributions are very welcome on T323750: Provide early feedback when a patch has job failures (feel free to tag T323750 with patches adding quibble.yaml to your project.)
🙌🏻 Thank you
- @hashar for code & architecture review, pushing through Jenkins configuration updates, and solving some low-level plumbing for T331061: Capture output from failed command and transmit to earlywarningbot
- @Urbanecm for fixing a bug where the bot reported on non-essential pipelines (T331236)
Cheers,
Kosta
[0] An alternative for getting real time progress is to watch Zuul TV
. There is also the excellent work in T214068: Display Zuul status of jobs for a change on Gerrit UI but this does not generate email/IRC notifications or set a verification label.[1] This will work for MediaWiki core, extensions, skins; in theory any CI job using Quibble could use it, though.
[2] Some jobs, like mwext-phan, won't report back early because they are not yet run via Quibble.