Existing tool:

• https://gerrit.wikimedia.org/r/admin/projects/mediawiki/tools/phpunit-patch-coverage
• https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-patch-docker/4623/console

Here's my understanding of what needs to be done:

1. Define an example sonar-project.properties file that projects can make use of.
   1. Identify and exclude rules that don't make sense for MediaWiki projects. For example the "Any string that is repeated more than once should be defined as a constant rule" will get flagged a ton of times unless we exclude it.
   2. Set up the default paths to analyze, i.e. includes, modules, define the tests root directory as tests, etc
2. Install the SonarQube Scanner for Jenkins
3. Install the Sonar gerrit plugin in Jenkins
4. Configure a Jenkins job to run sonar-scanner with Sonarcloud.io as the server (you need an API key, anyone in wmftest can generate it), and configure the Sonar gerrit plugin to post the results back to Gerrit. There is a bunch of configuration available with that but probably the default is fine for now.

If we want to import code coverage into sonarcloud.io we need to:

1. Generate the XML files with --log-junit and --coverage-clover, and copy both of those files somewhere.
2. When we send data to sonarcloud.io in step 5 above, we need to set CLI arguments to those files.
3. The files need to be cleaned up by a script to remove everything but the relevant data for the extension, and the paths inside the XML files may also need to be manually cleaned up.

I'm happy to help with a sample sonar-project.properties file and helping figure out how to work around the issues with importing the unit test XML and code coverage XML, but the rest is Jenkins config that someone else will have to do.

In T207046#4702909, @zeljkofilipin wrote:

• talk with @hashar about installing the SonarQube Scanner for Jenkins

The plugin is merely a Jenkins client for SonarQube. We would need a server installed somehow.

@hashar as far as I understood it, the client runs the analysis locally, and the server stores the data and creates reports. We can use SonarCloud for now as a server.

To demo SonarQube for the working group, I recommend to set one up on a WMCS instance. The process should be straightforward:

• create an instance
• install Jenkins sudo apt install jenkins
• add a web proxy to expose it publicly (iirc by default jenkins listens on port 8080
• connect to the instance, by default the first account gets admin privileges
• install the plugin, and configure it to use SonarCloud
• install the git plugin
• create a job that clone some repository and run the SonarQube plugin

I am not willing to add the plugin to the production Jenkins CI for a few reasons: I have no idea whether JJB supports it, that ties the CI service with a third party, the CI master is encumbered with too many plugins.

Change 472661 had a related patch set uploaded (by Gehel; owner: Gehel):
[integration/config@master] Add sonarcloud publication to maven site publish template

https://gerrit.wikimedia.org/r/472661

In T207046#4720115, @hashar wrote:

To demo SonarQube for the working group, I recommend to set one up on a WMCS instance.

There is already a lot that can be done with only very minimal integration with Jenkins. Publishing metrics to SonarCloud is just a matter of running an additional build command. It might require having the sonar analyzer in the docker image running the build (for non maven projects at least).

We can already have some basic reporting by checking the Sonar quality gates via API and failing the build job.

The best integration will come from using some kind of plugin, but we should not be blocked by that at this point.

@Gehel reported a Sonar bug https://community.sonarsource.com/t/sonar-scanner-should-honor-xdg-cache-home/4339

Change 474173 had a related patch set uploaded (by Hashar; owner: Gehel):
[integration/config@master] Add sonarcloud publication to maven site publish template

https://gerrit.wikimedia.org/r/474173

Change 472661 merged by jenkins-bot:
[integration/config@master] docker: set SONA_USER_HOME for SonarQube

https://gerrit.wikimedia.org/r/472661

Change 474173 merged by jenkins-bot:
[integration/config@master] Add sonarcloud publication to maven site publish template

https://gerrit.wikimedia.org/r/474173

Change 474660 had a related patch set uploaded (by Gehel; owner: Gehel):
[integration/config@master] rebuild wikidata-query-rdf docker image

https://gerrit.wikimedia.org/r/474660

Change 474660 merged by jenkins-bot:
[integration/config@master] rebuild wikidata-query-rdf docker image

https://gerrit.wikimedia.org/r/474660

Change 475470 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] Add configuration for running PHP code analysis on SonarQube

https://gerrit.wikimedia.org/r/475470

I've found a couple of bots so far, Reviewer-bot (Git/Reviewers and valhallasw/gerrit-reviewer-bot) and jenkins-bot, see 460516 for example.

There is also Cindy The Browser Test Bot.

@hashar to move this task forward, we need the sonar-scanner image built (gerrit/475496), and then we need to make adjustments to the post-merge step in order to use that image (gerrit/475470). Do you think you would have time to help us out with these two patches this week? I could meet today/tomorrow/Friday if you prefer to look at this together in a meeting. Thanks in advance!

gerrit review 12345,45 -m "some message"

https://gerrit.wikimedia.org/r/Documentation/rest-api-changes.html#set-review

Change 477618 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/WikimediaEvents@master] Do not merge: Add sonar-project.properties file

https://gerrit.wikimedia.org/r/477618

@zeljkofilipin @hashar @Gehel I updated the patch for adding the sonar scanner tool to the experimental build pipeline. I think it's getting closer but probably still needs some work.

Change 475470 merged by jenkins-bot:
[integration/config@master] Add job for running code analysis on SonarQube

https://gerrit.wikimedia.org/r/475470

Change 477618 abandoned by Zfilipin:
Do not merge: Add sonar-project.properties file

Reason:
WikimediaEvents does not use extension-quibble-composer template

https://gerrit.wikimedia.org/r/477618

Change 485018 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/Flow@master] WIP Add sonar-project.properties file

https://gerrit.wikimedia.org/r/485018

Change 485018 abandoned by Zfilipin:
WIP Add sonar-project.properties file

Reason:
Flow does not use extension-quibble-composer template 🤦‍♂️

https://gerrit.wikimedia.org/r/485018

Change 485025 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/WikibaseQuality@master] WIP Add sonar-project.properties file

https://gerrit.wikimedia.org/r/485025

I have tested wmf-sonar-scanner job at 485025 and it ran: wmf-sonar-scanner/1/console.

I was expecting WikibaseQuality to appear at https://sonarcloud.io/organizations/wmftest/projects but I don't see it there.

Change 485025 abandoned by Zfilipin:
WIP Add sonar-project.properties file

Reason:
just a test

https://gerrit.wikimedia.org/r/485025

Change 485025 restored by Kosta Harlan:
WIP Add sonar-project.properties file

https://gerrit.wikimedia.org/r/485025

Hmm. So I ran this locally with:

docker run -it --rm --entrypoint=sh --env-file=./.env -e SONAR_API_KEY=$SONAR_API_KEY -v $(pwd)/log:/log -v $(pwd)/ref:/srv/git:ro -v $(pwd):/src docker-registry.wikimedia.org/releng/java8-sonar-scanner:0.2.0

And then:

/run.sh  -Dsonar.projectKey=${ZUUL_PROJECT//\//-}  -Dsonar.host.url=https://sonarcloud.io  -Dsonar.projectName=${ZUUL_PROJECT//\//-}  -Dsonar.organization=wmftest

And it worked, see https://sonarcloud.io/dashboard?id=mediawiki-extensions-WikibaseQuality

I think we might need to modify the Docker image to run sonar-scanner in debug mode (-X) to see what the problem is.

Change 485025 abandoned by Zfilipin:
WIP Add sonar-project.properties file

Reason:
just a test

https://gerrit.wikimedia.org/r/485025