Page MenuHomePhabricator

Code health metrics spike
Closed, ResolvedPublic

Description

  • one repository (MobileFrontend, Minerva...)
  • one language (PHP, JavaScript...)
  • one metric (branch coverage...)
  • one tool (sonar cube...)
  • one feedback mechanism (bot leaving a reply in gerrit)

SonarQube runs for several Java repositories (see 474173 for the list) post-merge and sends the results to SonarCloud (see organizations/wmftest for results).

Done

  • 474660: rebuild wikidata-query-rdf docker image
  • 474173: Add sonarcloud publication to maven site publish template
  • 472661: docker: set SONA_USER_HOME for SonarQube
  • J133
  • 475470: Add configuration for running PHP code analysis on SonarQube
  • 477618: Do not merge: Add sonar-project.properties file
  • 485018: WIP Add sonar-project.properties file
  • 485025: WIP Add sonar-project.properties file

Event Timeline

kostajh claimed this task.Oct 22 2018, 2:35 PM
kostajh added a subscriber: kostajh.

I'll work on getting something set up this week.

Here's my understanding of what needs to be done:

  1. Define an example sonar-project.properties file that projects can make use of.
    1. Identify and exclude rules that don't make sense for MediaWiki projects. For example the "Any string that is repeated more than once should be defined as a constant rule" will get flagged a ton of times unless we exclude it.
    2. Set up the default paths to analyze, i.e. includes, modules, define the tests root directory as tests, etc
  2. Install the SonarQube Scanner for Jenkins
  3. Install the Sonar gerrit plugin in Jenkins
  4. Configure a Jenkins job to run sonar-scanner with Sonarcloud.io as the server (you need an API key, anyone in wmftest can generate it), and configure the Sonar gerrit plugin to post the results back to Gerrit. There is a bunch of configuration available with that but probably the default is fine for now.

If we want to import code coverage into sonarcloud.io we need to:

  1. Generate the XML files with --log-junit and --coverage-clover, and copy both of those files somewhere.
  2. When we send data to sonarcloud.io in step 5 above, we need to set CLI arguments to those files.
  3. The files need to be cleaned up by a script to remove everything but the relevant data for the extension, and the paths inside the XML files may also need to be manually cleaned up.

I'm happy to help with a sample sonar-project.properties file and helping figure out how to work around the issues with importing the unit test XML and code coverage XML, but the rest is Jenkins config that someone else will have to do.

kostajh removed kostajh as the assignee of this task.Oct 26 2018, 6:17 PM

And in the above, Sonarcloud.io can be replaced with our own hosted version of SonarQube, if we want to do that down the line.

zeljkofilipin added a subscriber: hashar.

Next steps:

  • talk with @hashar about installing the SonarQube Scanner for Jenkins
  • talk with @hashar about installing the SonarQube Scanner for Jenkins

The plugin is merely a Jenkins client for SonarQube. We would need a server installed somehow.

zeljkofilipin triaged this task as Normal priority.Oct 29 2018, 4:05 PM
zeljkofilipin moved this task from Backlog 🔙 to In Progress 🔨 on the User-zeljkofilipin board.

@hashar as far as I understood it, the client runs the analysis locally, and the server stores the data and creates reports. We can use SonarCloud for now as a server.

hashar added a comment.Nov 5 2018, 1:39 PM

To demo SonarQube for the working group, I recommend to set one up on a WMCS instance. The process should be straightforward:

  • create an instance
  • install Jenkins sudo apt install jenkins
  • add a web proxy to expose it publicly (iirc by default jenkins listens on port 8080
  • connect to the instance, by default the first account gets admin privileges
  • install the plugin, and configure it to use SonarCloud
  • install the git plugin
  • create a job that clone some repository and run the SonarQube plugin

I am not willing to add the plugin to the production Jenkins CI for a few reasons: I have no idea whether JJB supports it, that ties the CI service with a third party, the CI master is encumbered with too many plugins.

Change 472661 had a related patch set uploaded (by Gehel; owner: Gehel):
[integration/config@master] Add sonarcloud publication to maven site publish template

https://gerrit.wikimedia.org/r/472661

Gehel added a subscriber: Gehel.Nov 9 2018, 5:17 PM

To demo SonarQube for the working group, I recommend to set one up on a WMCS instance.

There is already a lot that can be done with only very minimal integration with Jenkins. Publishing metrics to SonarCloud is just a matter of running an additional build command. It might require having the sonar analyzer in the docker image running the build (for non maven projects at least).

We can already have some basic reporting by checking the Sonar quality gates via API and failing the build job.

The best integration will come from using some kind of plugin, but we should not be blocked by that at this point.

Change 474173 had a related patch set uploaded (by Hashar; owner: Gehel):
[integration/config@master] Add sonarcloud publication to maven site publish template

https://gerrit.wikimedia.org/r/474173

Change 472661 merged by jenkins-bot:
[integration/config@master] docker: set SONA_USER_HOME for SonarQube

https://gerrit.wikimedia.org/r/472661

Change 474173 merged by jenkins-bot:
[integration/config@master] Add sonarcloud publication to maven site publish template

https://gerrit.wikimedia.org/r/474173

Change 474660 had a related patch set uploaded (by Gehel; owner: Gehel):
[integration/config@master] rebuild wikidata-query-rdf docker image

https://gerrit.wikimedia.org/r/474660

Change 474660 merged by jenkins-bot:
[integration/config@master] rebuild wikidata-query-rdf docker image

https://gerrit.wikimedia.org/r/474660

Change 475470 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] Add configuration for running PHP code analysis on SonarQube

https://gerrit.wikimedia.org/r/475470

zeljkofilipin updated the task description. (Show Details)
zeljkofilipin added a comment.EditedNov 26 2018, 1:57 PM

I've found a couple of bots so far, Reviewer-bot (Git/Reviewers and valhallasw/gerrit-reviewer-bot) and jenkins-bot, see 460516 for example.

@hashar to move this task forward, we need the sonar-scanner image built (gerrit/475496), and then we need to make adjustments to the post-merge step in order to use that image (gerrit/475470). Do you think you would have time to help us out with these two patches this week? I could meet today/tomorrow/Friday if you prefer to look at this together in a meeting. Thanks in advance!

Change 477618 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/WikimediaEvents@master] Do not merge: Add sonar-project.properties file

https://gerrit.wikimedia.org/r/477618

zeljkofilipin removed zeljkofilipin as the assignee of this task.Dec 17 2018, 5:32 PM

Not working on this at the moment. Will probably continue after T206663 is resolved.

@zeljkofilipin @hashar @Gehel I updated the patch for adding the sonar scanner tool to the experimental build pipeline. I think it's getting closer but probably still needs some work.

Change 475470 merged by jenkins-bot:
[integration/config@master] Add job for running code analysis on SonarQube

https://gerrit.wikimedia.org/r/475470

Change 477618 abandoned by Zfilipin:
Do not merge: Add sonar-project.properties file

Reason:
WikimediaEvents does not use extension-quibble-composer template

https://gerrit.wikimedia.org/r/477618

Change 485018 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/Flow@master] WIP Add sonar-project.properties file

https://gerrit.wikimedia.org/r/485018

zeljkofilipin updated the task description. (Show Details)

Change 485018 abandoned by Zfilipin:
WIP Add sonar-project.properties file

Reason:
Flow does not use extension-quibble-composer template 🤦‍♂️

https://gerrit.wikimedia.org/r/485018

Change 485025 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/WikibaseQuality@master] WIP Add sonar-project.properties file

https://gerrit.wikimedia.org/r/485025

I have tested wmf-sonar-scanner job at 485025 and it ran: wmf-sonar-scanner/1/console.

I was expecting WikibaseQuality to appear at https://sonarcloud.io/organizations/wmftest/projects but I don't see it there.

zeljkofilipin removed zeljkofilipin as the assignee of this task.Jan 22 2019, 1:12 PM

I'm not working on this at the moment, so removing myself as assignee. I will probably continue after All Hands, if somebody else doesn't take it.

Change 485025 abandoned by Zfilipin:
WIP Add sonar-project.properties file

Reason:
just a test

https://gerrit.wikimedia.org/r/485025

zeljkofilipin updated the task description. (Show Details)
zeljkofilipin removed a project: Patch-For-Review.
hashar removed a subscriber: hashar.Jan 24 2019, 10:57 AM

Change 485025 restored by Kosta Harlan:
WIP Add sonar-project.properties file

https://gerrit.wikimedia.org/r/485025

Hmm. So I ran this locally with:

docker run -it --rm --entrypoint=sh --env-file=./.env -e SONAR_API_KEY=$SONAR_API_KEY -v $(pwd)/log:/log -v $(pwd)/ref:/srv/git:ro -v $(pwd):/src docker-registry.wikimedia.org/releng/java8-sonar-scanner:0.2.0

And then:

/run.sh  -Dsonar.projectKey=${ZUUL_PROJECT//\//-}  -Dsonar.host.url=https://sonarcloud.io  -Dsonar.projectName=${ZUUL_PROJECT//\//-}  -Dsonar.organization=wmftest

And it worked, see https://sonarcloud.io/dashboard?id=mediawiki-extensions-WikibaseQuality

I think we might need to modify the Docker image to run sonar-scanner in debug mode (-X) to see what the problem is.

Change 485025 abandoned by Zfilipin:
WIP Add sonar-project.properties file

Reason:
just a test

https://gerrit.wikimedia.org/r/485025

zeljkofilipin closed this task as Resolved.Feb 25 2019, 12:45 PM
zeljkofilipin claimed this task.

As far as I can see, this is resolved.

Example:

Next steps: