Page MenuHomePhabricator

Code coverage does not import cleanly into SonarQube
Closed, ResolvedPublic

Description

Steps to reproduce:

  1. Set up https://github.com/addshore/mediawiki-docker-dev and install WikimediaEvents extension
  2. mw-docker-dev phpunit-file default extensions/WikimediaEvents/tests/phpunit --coverage-clover /var/www/mediawiki/docs/code-coverage/coverage.xml --log-junit /var/www/mediawiki/docs/unit.xml to generate the coverage file and the unit test run output file. Files are attached.
    1. Side note: if you haven't modified Mediawiki core's suite.xml file, you'll wait a long time for coverage to be generated and it won't include WikimediaEvents. You need to modify suite.xml to point to WikimediaEvents
  3. Run sonar-scanner from within WikimediaEvents directory: sonar-scanner -Dsonar.projectKey=WikimediaEvents -Dsonar.organization=wmftest -Dsonar.php.tests.reportPath=../../docs/unit.xml -Dsonar.php.coverage.reportPaths=../../docs/code-coverage/coverage.xml -Dsonar.tests=tests -Dsonar.sources=includes,modules -Dsonar.host.url=https://sonarcloud.io -Dsonar.login=REDACTED

In the output you'll see:

13:01:45.583 INFO: Analyzing PHPUnit test report: ../../docs/unit.xml
13:01:45.712 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "no event" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "wrong type" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "right channel" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "old channel" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "wrong channel" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "simple" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "type" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "type fallback" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "success" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "failure" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "success with status" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "failure with status" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.713 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "Status, good" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "Status, bad" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "Status, multiple" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : AuthManagerStatsdHandlerTest.testHandle with data set "StatusValue" in AuthManagerStatsdHandlerTest::testHandle
13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testHashSensitiveQueryParams with data set #0 in WikimediaEvents\Tests\PageViewsTest::testHashSensitiveQueryParams
13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testHashSensitiveQueryParams with data set #1 in WikimediaEvents\Tests\PageViewsTest::testHashSensitiveQueryParams
13:01:45.714 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testHashSensitiveQueryParams with data set #2 in WikimediaEvents\Tests\PageViewsTest::testHashSensitiveQueryParams
13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #0 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData
13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #1 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData
13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #2 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData
13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #3 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData
13:01:45.715 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #4 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData
13:01:45.716 WARN: Test cases must always be descendants of a file-based suite, skipping : WikimediaEvents\Tests\PageViewsTest.testRedactSensitiveData with data set #5 in WikimediaEvents\Tests\PageViewsTest::testRedactSensitiveData
13:01:45.723 DEBUG: Following file is not located in the test folder specified in the Sonar configuration: /var/www/mediawiki/extensions/WikimediaEvents/tests/phpunit/AuthManagerStatsdHandlerTest.php. The test results won't be reported in Sonar.
13:01:45.724 DEBUG: Following file is not located in the test folder specified in the Sonar configuration: /var/www/mediawiki/extensions/WikimediaEvents/tests/phpunit/PageViewsTest.php. The test results won't be reported in Sonar.
13:01:45.728 INFO: Analyzing PHPUnit coverage report: ../../docs/code-coverage/coverage.xml
13:01:45.728 DEBUG: Parsing file: /Users/kharlan/src/mediawiki/extensions/WikimediaEvents/../../docs/code-coverage/coverage.xml
13:01:45.765 WARN: Could not resolve 3 file paths in coverage.xml, first unresolved path: /var/www/mediawiki/extensions/WikimediaEvents/includes/AuthManagerStatsdHandler.php
13:01:45.765 INFO: Sensor PHP sensor [php] (done) | time=2647ms

I found https://github.com/Codeception/Codeception/issues/5004, and after reading that I was able to modify by hand the generated XML so that "test cases would be descendants of a file-based suite", and that's why you can see code coverage for WikimediaEvents on sonarcloud.io, but otherwise it does not work.

Event Timeline

MobileFrontend generates code coverage reports, so we can use that as an example for JS.

Change 494548 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] Remove requirement for properties file, run and import coverage

https://gerrit.wikimedia.org/r/494548

Change 494548 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] Remove requirement for properties file, import coverage if present

https://gerrit.wikimedia.org/r/494548

Change 494548 merged by jenkins-bot:
[integration/config@master] Remove requirement for properties file, import coverage if present

https://gerrit.wikimedia.org/r/494548

Change 508019 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] Generate junit.xml for sonar-scanner's usage

https://gerrit.wikimedia.org/r/508019

Change 508019 merged by jenkins-bot:
[integration/config@master] Generate junit.xml for sonar-scanner's usage

https://gerrit.wikimedia.org/r/508019

kostajh claimed this task.

As far as I can see, this is no longer a problem.