Page MenuHomePhabricator

WikibaseQualityConstraints coverage reports broken
Open, Needs TriagePublic

Description

On https://doc.wikimedia.org/cover-extensions/, WikibaseQualityConstraints doesn’t report any coverage:

Following the link to the extension’s coverage report yields “Empty directory!”

This is a bit sad, especially since WikibaseQualityConstraints used to be one of the extensions with the highest reported coverage (above 90%, if memory serves).

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 10 2020, 5:24 PM

Running php tests/phpunit/phpunit.php -c extensions/WikibaseQualityConstraints/phpunit.xml.dist in MediaWiki core still produces a valid coverage report in extensions/WikibaseQualityConstraints/tests/coverage/ (not quite as good as I remembered, 82% covered lines overall).

Does not run extension-coverage according to layout.yaml while the other from screenshot does.

Change 563573 had a related patch set uploaded (by Umherirrender; owner: Umherirrender):
[integration/config@master] Add extension-coverage to pages from cover-extensions

https://gerrit.wikimedia.org/r/563573

Change 563573 merged by jenkins-bot:
[integration/config@master] layout: Restore extension-coverage to projects with …-codehealth

https://gerrit.wikimedia.org/r/563573

Mentioned in SAL (#wikimedia-releng) [2020-01-10T23:22:25Z] <James_F> Zuul: Restore extension-coverage to projects with …-codehealth T242444

This should fix itself the next time something successfully merges in each of the affected repos.

After today’s l10n-bot change, WikibaseQualityConstraints has a proper list entry again –

– but following the link still results in “Empty directory!” At least Wikibase and WikibaseMediaInfo seem to have the same problem, though.

I have seen a failure on the postmerge
https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-docker-publish/16752/console

Not sure, but the cause looks like this line:

19:52:22 + php -d zend_extension=xdebug.so /workspace/src/tests/phpunit/phpunit.php --testsuite extensions --coverage-clover /workspace/log/clover.xml --coverage-html /workspace/cover --log-junit /workspace/log/junit.xml /workspace/src/extensions/Wikibase/tests/phpunit
19:52:22 Using PHP 7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1
19:52:22 Cannot open file "/workspace/src/extensions/Wikibase/tests/phpunit.php".

That postmerge failure is also tracked as T242617, FWIW.

Addshore added a subscriber: Addshore.EditedJan 30 2020, 8:27 AM

I have seen a failure on the postmerge
https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-docker-publish/16752/console

Not sure, but the cause looks like this line:

19:52:22 + php -d zend_extension=xdebug.so /workspace/src/tests/phpunit/phpunit.php --testsuite extensions --coverage-clover /workspace/log/clover.xml --coverage-html /workspace/cover --log-junit /workspace/log/junit.xml /workspace/src/extensions/Wikibase/tests/phpunit
19:52:22 Using PHP 7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1
19:52:22 Cannot open file "/workspace/src/extensions/Wikibase/tests/phpunit.php".

Wikibase/tests/phpunit.php does indeed not exist, and it never has (in my memory).
Why is this job trying to run phpunit in this way instead of the same way done in the other CI jobs?
I wonder if there would be any way to do this within quibble?
Not sure if we should discuss this here or on T242617

Change 583677 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[integration/config@master] Add extension-coverage to WikibaseQualityConstraints

https://gerrit.wikimedia.org/r/583677

Change 583677 merged by jenkins-bot:
[integration/config@master] Add extension-coverage to WikibaseQualityConstraints

https://gerrit.wikimedia.org/r/583677

Maybe the above change fixes it – Iaf27c5ac36 would be one change that could be merged to trigger a post-merge in the repository and test this, if anyone wants to review it. (I think it’s a little bit too substantial for a self-merge, but it’s not huge either.)

hashar added a subscriber: hashar.Thu, Jul 30, 12:41 PM

Tentatively the code coverage report is handled by the codehealth pipeline (T224990). But some repositories also have the extension-coverage template and the two do not play nices when run together leading to the coverage report eventually being empty: T259109

Change 617453 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Avoid duplicate codecoverage report

https://gerrit.wikimedia.org/r/617453

Change 617453 abandoned by Hashar:
[integration/config@master] Avoid duplicate codecoverage report

Reason:
Will do it the other way around, namely stop publishing from the codehealth job (essentially reverting https://gerrit.wikimedia.org/r/c/integration/config/ /514016 ).

https://gerrit.wikimedia.org/r/617453

Change 619370 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Revert "Publish coverage reports from extension-codehealth"

https://gerrit.wikimedia.org/r/619370