Running php tests/phpunit/phpunit.php -c extensions/WikibaseQualityConstraints/phpunit.xml.dist in MediaWiki core still produces a valid coverage report in extensions/WikibaseQualityConstraints/tests/coverage/ (not quite as good as I remembered, 82% covered lines overall).

Does not run extension-coverage according to layout.yaml while the other from screenshot does.

Change 563573 had a related patch set uploaded (by Umherirrender; owner: Umherirrender):
[integration/config@master] Add extension-coverage to pages from cover-extensions

https://gerrit.wikimedia.org/r/563573

Change 563573 merged by jenkins-bot:
[integration/config@master] layout: Restore extension-coverage to projects with …-codehealth

https://gerrit.wikimedia.org/r/563573

Mentioned in SAL (#wikimedia-releng) [2020-01-10T23:22:25Z] <James_F> Zuul: Restore extension-coverage to projects with …-codehealth T242444

This should fix itself the next time something successfully merges in each of the affected repos.

After today’s l10n-bot change, WikibaseQualityConstraints has a proper list entry again –

– but following the link still results in “Empty directory!” At least Wikibase and WikibaseMediaInfo seem to have the same problem, though.

I have seen a failure on the postmerge
https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-docker-publish/16752/console

Not sure, but the cause looks like this line:

19:52:22 + php -d zend_extension=xdebug.so /workspace/src/tests/phpunit/phpunit.php --testsuite extensions --coverage-clover /workspace/log/clover.xml --coverage-html /workspace/cover --log-junit /workspace/log/junit.xml /workspace/src/extensions/Wikibase/tests/phpunit
19:52:22 Using PHP 7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1
19:52:22 Cannot open file "/workspace/src/extensions/Wikibase/tests/phpunit.php".

That postmerge failure is also tracked as T242617, FWIW.

In T242444#5802995, @Umherirrender wrote:

I have seen a failure on the postmerge
https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-docker-publish/16752/console

Not sure, but the cause looks like this line:

19:52:22 + php -d zend_extension=xdebug.so /workspace/src/tests/phpunit/phpunit.php --testsuite extensions --coverage-clover /workspace/log/clover.xml --coverage-html /workspace/cover --log-junit /workspace/log/junit.xml /workspace/src/extensions/Wikibase/tests/phpunit
19:52:22 Using PHP 7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1
19:52:22 Cannot open file "/workspace/src/extensions/Wikibase/tests/phpunit.php".

Wikibase/tests/phpunit.php does indeed not exist, and it never has (in my memory).
Why is this job trying to run phpunit in this way instead of the same way done in the other CI jobs?
I wonder if there would be any way to do this within quibble?
Not sure if we should discuss this here or on T242617

Change 583677 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[integration/config@master] Add extension-coverage to WikibaseQualityConstraints

https://gerrit.wikimedia.org/r/583677

Change 583677 merged by jenkins-bot:
[integration/config@master] Add extension-coverage to WikibaseQualityConstraints

https://gerrit.wikimedia.org/r/583677

Maybe the above change fixes it – Iaf27c5ac36 would be one change that could be merged to trigger a post-merge in the repository and test this, if anyone wants to review it. (I think it’s a little bit too substantial for a self-merge, but it’s not huge either.)

Tentatively the code coverage report is handled by the codehealth pipeline (T224990). But some repositories also have the extension-coverage template and the two do not play nices when run together leading to the coverage report eventually being empty: T259109

Change 617453 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Avoid duplicate codecoverage report

https://gerrit.wikimedia.org/r/617453

Change 617453 abandoned by Hashar:
[integration/config@master] Avoid duplicate codecoverage report

Reason:
Will do it the other way around, namely stop publishing from the codehealth job (essentially reverting https://gerrit.wikimedia.org/r/c/integration/config/ /514016 ).

https://gerrit.wikimedia.org/r/617453

Change 619370 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Revert "Publish coverage reports from extension-codehealth"

https://gerrit.wikimedia.org/r/619370

Change 619987 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/WikibaseQualityConstraints@master] Rename Unit testing directory to unit

https://gerrit.wikimedia.org/r/619987

Change 619987 abandoned by Kosta Harlan:
[mediawiki/extensions/WikibaseQualityConstraints@master] Rename Unit testing directory to unit

Reason:
Oops, this was off-base. "Unit" is explicitly handled by phpunit.xml.dist and the error message I got when running these locally is a bit different from what is reported in the task.

https://gerrit.wikimedia.org/r/619987

Change 619370 merged by jenkins-bot:
[integration/config@master] Revert "Publish coverage reports from extension-codehealth"

https://gerrit.wikimedia.org/r/619370

I looked at extensions that lacked a coverage report by looking on doc1001.eqiad.wmnet in /srv/docroot/org/wikimedia/doc/cover-extensions:

WikiLambda WikibaseManifest Wikibase Popups MachineVision

Regenerated them with:

zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/MachineVision --change 623506,1
zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/Popups --change 624137,2
zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/Wikibase --change 625825,1
zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/WikibaseManifest --change 625068,1
zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/WikiLambda --change 625563,1

MachineVision and Wikibase do not have the job mwext-phpunit-coverage-docker-publish