Rephrasing bug to generalize the idea. I agree it would be nice if one wouldn't have to dig into Jenkins to find out what the problem is.

On the other hand, the Jenkins interface is very simple to use, and especially the PHPUnit module is very well integrated. From the parent job build page to the exact name of the failing test is only 2 clicks from the comment in Gerrit.

1. Gerrit comment: https://integration.mediawiki.org/ci/job/MediaWiki-GIT-Fetching/1104/
2. Failing build: MediaWiki-Tests-Misc
3. https://integration.mediawiki.org/ci/job/MediaWiki-Tests-Misc/841/
4. Result:

Test Result (1 failure / +1)

• WikiPageTest.testDoRollback

When the parent job "GIT-Fetching" is finished with all children builds, it does the Verified -1/+1 in Gerrit and then either leaves a comment in Gerrit that the build has PASSED with a link to the build page in Jenkins (nothing more - if one wants to know the results of the lint check, or individual phpunit tests, style test, testswarm etc. - then one should be able to get there via the build page in Jenkins).

Idea: In case the build fails, let Jenkins leave a little more elaborate comment in Gerrit.

• Indicate which child build failed (lint, style, phpunit, testswarm, ..)
• Include the list (up to 10?) of names of tests that failed

I'd rather not add a workaround to add something in the Gerrit comment maker for one particular unit test only (bug bug 35942 in this case). That means we'd have to hard code the name of the unit test somewhere and extract the result from the phpunit build results.

I don't know how easy it is to extract these stats from where the Gerrit comment is formatted, but it seems for phpunit jobs the information is very well aggregated, something like this maybe?

(jenkins-bot)

Build PASSED
Details: https://integration.mediawiki.org/ci/job/MediaWiki-GIT-Fetching/1138/

(jenkins-bot)

Build FAILED
Aggregated Test Result: 1 failure

Failed build(s):

• MediaWiki-Tests-Misc: 1 failure
  • LanguageTest.testMissingMessageEntries

Details: https://integration.mediawiki.org/ci/job/MediaWiki-GIT-Fetching/1104/

(jenkins-bot)

Build FAILED
Aggregated Test Result: 3 failures

Failed build(s):

• MediaWiki-Tests-Misc: 1 failure
  • LanguageTest.testMissingMessageEntries
• MediaWiki-Tests-TestSwarm: 2 failures
  • http://integration.mediawiki.org/testswarm/job/0000/

Details: https://integration.mediawiki.org/ci/job/MediaWiki-GIT-Fetching/0000/

Maybe limit to only show the first 10.

This is mostly fulfilled by Zuul which have deployed in November 2012. We can tweak each job URL to point to something more useful than the console.