Steps to reproduce:
- Set up https://github.com/addshore/mediawiki-docker-dev and install WikimediaEvents extension
- mw-docker-dev phpunit-file default extensions/WikimediaEvents/tests/phpunit --coverage-clover /var/www/mediawiki/docs/code-coverage/coverage.xml --log-junit /var/www/mediawiki/docs/unit.xml to generate the coverage file and the unit test run output file. Files are attached.
- Side note: if you haven't modified Mediawiki core's suite.xml file, you'll wait a long time for coverage to be generated and it won't include WikimediaEvents. You need to modify suite.xml to point to WikimediaEvents
- Run sonar-scanner from within WikimediaEvents directory: sonar-scanner -Dsonar.projectKey=WikimediaEvents -Dsonar.organization=wmftest -Dsonar.php.tests.reportPath=../../docs/unit.xml -Dsonar.php.coverage.reportPaths=../../docs/code-coverage/coverage.xml -Dsonar.tests=tests -Dsonar.sources=includes,modules -Dsonar.host.url=https://sonarcloud.io -Dsonar.login=REDACTED
In the output you'll see:
13:01:45.583 INFO: Analyzing PHPUnit test report: ../../docs/unit.xml 13:01:45.712 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "no event" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "wrong type" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "right channel" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "old channel" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "wrong channel" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "simple" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "type" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "type fallback" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "success" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "failure" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "success with status" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "failure with status" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "Status, good" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "Status, bad" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "Status, multiple" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "StatusValue" in AuthManagerStatsdHandlerTest::testHandle 13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testHashSensitiveQueryParams with data set #0 in WikimediaEvents\Tests\PageViewsTest::testHashSensitiveQueryParams 13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testHashSensitiveQueryParams with data set #1 in WikimediaEvents\Tests\PageViewsTest::testHashSensitiveQueryParams 13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testHashSensitiveQueryParams with data set #2 in WikimediaEvents\Tests\PageViewsTest::testHashSensitiveQueryParams 13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #0 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData 13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #1 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData 13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #2 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData 13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #3 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData 13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #4 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData 13:01:45.716 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #5 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData 13:01:45.723 DEBUG: Following file is not located in the test folder specified in the Sonar configuration: /var/www/mediawiki/extensions/WikimediaEvents/tests/phpunit/AuthManagerStatsdHandlerTest.php. The test results won't be reported in Sonar. 13:01:45.724 DEBUG: Following file is not located in the test folder specified in the Sonar configuration: /var/www/mediawiki/extensions/WikimediaEvents/tests/phpunit/PageViewsTest.php. The test results won't be reported in Sonar. 13:01:45.728 INFO: Analyzing PHPUnit coverage report: ../../docs/code-coverage/coverage.xml 13:01:45.728 DEBUG: Parsing file: /Users/kharlan/src/mediawiki/extensions/WikimediaEvents/../../docs/code-coverage/coverage.xml 13:01:45.765 WARN: Could not resolve 3 file paths in coverage.xml, first unresolved path: /var/www/mediawiki/extensions/WikimediaEvents/includes/AuthManagerStatsdHandler.php 13:01:45.765 INFO: Sensor PHP sensor [php] (done) | time=2647ms
I found https://github.com/Codeception/Codeception/issues/5004, and after reading that I was able to modify by hand the generated XML so that "test cases would be descendants of a file-based suite", and that's why you can see code coverage for WikimediaEvents on sonarcloud.io, but otherwise it does not work.