Page MenuHomePhabricator

Report results from SonarCloud to Gerrit
Open, MediumPublic

Description

This will be implemented with an app on toolforge (https://tools.wmflabs.org/sonarqubebot/)

Implementation:

  • Application to listen to POST requests from SonarQube
    • Validate that the requests come from SonarQube and not some rando
  • Create a SonarQubeBot user in gerrit
  • Add SonarQubeBot user to stream-events group
  • Craft a comment and post to the gerrit patchset

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 25 2019, 11:20 AM
zeljkofilipin triaged this task as Medium priority.Feb 25 2019, 12:30 PM
zeljkofilipin renamed this task from Report results from sonar cloud to gerrit to Report results from SonarCloud to Gerrit.Feb 25 2019, 12:45 PM
zeljkofilipin moved this task from Backlog 🔙 to Next 🔜 on the User-zeljkofilipin board.
kostajh added a subscriber: kostajh.Mar 5 2019, 1:53 AM

That seems OK as long as the URL includes the branch (Gerrit patchset number) so that clicking the link takes you to a relevant page on SonarCloud.

Later on it would probably make sense to implement a webhook for reporting: https://docs.sonarqube.org/latest/project-administration/webhooks/

@zeljkofilipin I'm interested to pursue setting up a simple tool on ToolForge that:

  1. Is configured to listen for webhook data from SonarQube
  2. Is responsible for posting a comment in Gerrit with the link to the build, the pass/fail status, and (maybe) a quick summary of the new issues found

If you don't have time to work on this in the next two weeks, I might get started with that process, let me know what you think please.

I really don't know how much time I'll have in the next week or two. If you have the time, go ahead. Toolforge tool sounds like a good idea.

Real-world POST from the SonarCloud webhook:

 json
{
  "serverUrl": "https://sonarcloud.io",
  "taskId": "AWm6bF2mtK8xldclsL0c",
  "status": "SUCCESS",
  "analysedAt": "2019-03-26T15:31:29+0100",
  "changedAt": "2019-03-26T15:31:29+0100",
  "project": {
    "key": "mediawiki-core",
    "name": "mediawiki-core",
    "url": "https://sonarcloud.io/dashboard?id=mediawiki-core"
  },
  "branch": {
    "name": "490363",
    "type": "SHORT",
    "isMain": false,
    "url": "https://sonarcloud.io/project/issues?branch=490363&id=mediawiki-core&resolved=false"
  },
  "qualityGate": {
    "name": "Sonar way",
    "status": "ERROR",
    "conditions": [
      {
        "metric": "new_reliability_rating",
        "operator": "GREATER_THAN",
        "value": "3",
        "status": "ERROR",
        "errorThreshold": "1"
      },
      {
        "metric": "new_security_rating",
        "operator": "GREATER_THAN",
        "value": "2",
        "status": "ERROR",
        "errorThreshold": "1"
      },
      {
        "metric": "new_maintainability_rating",
        "operator": "GREATER_THAN",
        "value": "1",
        "status": "OK",
        "errorThreshold": "1"
      },
      {
        "metric": "new_coverage",
        "operator": "LESS_THAN",
        "status": "NO_VALUE",
        "errorThreshold": "80"
      },
      {
        "metric": "new_duplicated_lines_density",
        "operator": "GREATER_THAN",
        "status": "NO_VALUE",
        "errorThreshold": "3"
      }
    ]
  },
  "properties": {}
}
kostajh claimed this task.Mar 26 2019, 5:36 PM
kostajh moved this task from Backlog to In-Progress on the Code-Health-Metrics board.
kostajh updated the task description. (Show Details)Mar 26 2019, 9:24 PM
kostajh added a subscriber: mmodell.

@mmodell could you possibly help with "Create a SonarQubeBot user in gerrit" and "Add SonarQubeBot user to stream-events group" from the above, or could you point me to someone who could?

I think @thcipriani is working on cleaning up Gerrit groups at the moment. He might be able to help.

I met with @thcipriani about this today. In the short-term what we'll do is:

  1. Modify the wmf-sonar-scanner-{name} job template to poll for analysis completion

In the java8-sonar-scanner job we'll want to pipe the output to /log/scanner-output.txt

Then we'll add a shell script step which will parse that output:

- job-template:
    name: 'wmf-sonar-scanner-{name}'
    node: DebianJessieDocker
    concurrent: false
    branch: '$ZUUL_BRANCH'
    properties:
     - build-discarder:
         days-to-keep: 15
    triggers:
     - zuul
    builders:
    - docker-log-dir
    - docker-src-dir
    - docker-cache-dir
    - docker-ci-src-setup-simple
    - docker-run-with-log-cache-src:
       image: 'docker-registry.wikimedia.org/releng/java8-sonar-scanner:0.4.0'
       logdir: '/log'
       args: |
         -Dsonar.projectKey={projectname} \
         -Dsonar.projectName={projectname} \
         -Dsonar.organization=wmftest \
         -Dsonar.host.url=https://sonarcloud.io \
         -Dsonar.branch.target="$ZUUL_BRANCH" \
         -Dsonar.branch.name={branch} \
         -X
    - shell: |
    # Code goes here

In the "code goes here" section we will look for a line like this INFO: More about the report processing at https://sonarcloud.io/api/ce/task?id=AWm7SW1T9XBIOzEJZvP0.

We will need to extract the ID from that string, then we can set up a polling script that calls https://sonarcloud.io/web_api/api/ce?query=task with the ID, say, every 30 seconds for 5 minutes.

Once we see that the task has completed (status in the API response) we can make another API call to the quality gates endpoint (https://sonarcloud.io/web_api/api/qualitygates?query=analysisId). That gives us a JSON document we can dump to STDOUT, and also if status in that document is ERROR than we can return a non-zero exit code so the Jenkins job shows a failure. We can also output a link to the branch in SonarQube using this format https://sonarcloud.io/dashboard?branch={gerritChangeNumber}&id={ProjectKey}, e.g. https://sonarcloud.io/dashboard?branch=490363&id=mediawiki-core

I forgot to mention; being able to post meaningful comments on the task with information about issues uncovered, for example, is going to take longer as we need new infrastructure setup. One idea @thcipriani had was a Jenkins user that can build jobs and comment in gerrit; have the SonarQube webhook ping Jenkins with a token, and have that trigger a job build which accesses the POSTed data from SonarQube and post a comment with that info to gerrit.

Change 501465 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] wmf-sonar-scanner: Report back quality gate result using polling

https://gerrit.wikimedia.org/r/501465

Change 501465 merged by jenkins-bot:
[integration/config@master] wmf-sonar-scanner: Report back quality gate result using polling

https://gerrit.wikimedia.org/r/501465

kostajh moved this task from In-Progress to Done on the Code-Health-Metrics board.May 14 2019, 2:07 PM
kostajh moved this task from Done to Backlog on the Code-Health-Metrics board.Jul 1 2019, 2:32 PM

@thcipriani wondering if the code health group could move ahead with this. We have the polling script approach working but we would like to use a bot so that we can post a small summary of the specific codehealth issues (rather than just a link to sonarcloud). I know you had also mentioned triggering a job build directly from the webhook. Please let us know if you have thoughts on a preferred approach.

@thcipriani wondering if the code health group could move ahead with this. We have the polling script approach working but we would like to use a bot so that we can post a small summary of the specific codehealth issues (rather than just a link to sonarcloud). I know you had also mentioned triggering a job build directly from the webhook. Please let us know if you have thoughts on a preferred approach.

Hrm, so I mentioned triggering a job via the webhook/remote token, but that's not currently well supported by our setup; i.e., you would have to add an auth-token to a job definition and we don't currently have a sane way to keep that info private. Plus it's a very Jenkins-specific approach, which may not be very future-proof.

The polling setup is non-optimal since it ties up a worker while it's running; however, that currently seems like the least bad option. We could mitigate the impact of tying up a worker machine by having the current job trigger a job that does the polling and posting to Gerrit on a dedicated, small, instance. We could create an integration machine for this purpose without a lot of resources (or use the existing trigger machine).

We'd still need a Gerrit user that can comment on tasks (can be setup as a bot account on Wikitech), put the creds in our Jenkins somewhere (you'd need a relenger's help for that), have the current job trigger a polling job, poll until complete, then trigger a script that parses the output and comments on Gerrit.

Posting comments on Gerrit through the change API ( https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html ) isn't too bad -- the API is one of Gerrit's strong points (I wrote a library in python with the idea to do this at one point https://github.com/thcipriani/grrit -- might be helpful).

Change 557001 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] jjb: Modify branch name passed to sonar-scanner

https://gerrit.wikimedia.org/r/557001

kostajh updated the task description. (Show Details)Dec 13 2019, 2:07 PM
kostajh updated the task description. (Show Details)

I've updated https://github.com/kostajh/sonarqubebot so that a more informative message will be posted (example https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Echo/+/556830#message-df768acbc9f195ec681804105a8e4906c3baf584) and will follow up later with adding inline code comments for specific violations.

Change 557001 merged by jenkins-bot:
[integration/config@master] jjb: Modify branch name passed to sonar-scanner

https://gerrit.wikimedia.org/r/557001

Change 557138 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] jjb: Add back codehealth messages and adjust success/failure pattern

https://gerrit.wikimedia.org/r/557138

Change 557138 merged by jenkins-bot:
[integration/config@master] jjb: Add back codehealth messages and adjust success/failure pattern

https://gerrit.wikimedia.org/r/557138

Change 559419 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] Codehealth pipeline: Don't report back to gerrit

https://gerrit.wikimedia.org/r/559419

Change 559419 merged by jenkins-bot:
[integration/config@master] Codehealth pipeline: Don't report back to gerrit

https://gerrit.wikimedia.org/r/559419

Mentioned in SAL (#wikimedia-releng) [2019-12-19T09:40:59Z] <hashar> Deploying Zuul change "Codehealth pipeline: Don't report back to gerrit" https://gerrit.wikimedia.org/r/559419 for T217008

Change 559441 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] dockerfiles: Drop poll-sonar-for-response script

https://gerrit.wikimedia.org/r/559441

Change 559441 merged by jenkins-bot:
[integration/config@master] dockerfiles: Drop poll-sonar-for-response script

https://gerrit.wikimedia.org/r/559441