Hey @MSantos can you fill the task description with more detail? Maybe using the template?

@MSantos T238004 is not yet live but it has all the patches which show what you'd want to do. Are those patches something you or others from #product-infrastructure-team-backlog would work on or would you like someone from Code-Health-Metrics to lead? I can't promise it would happen soon but I think we could do it this quarter sometime.

Change 606241 had a related patch set uploaded (by MSantos; owner: MSantos):
[integration/config@master] Add run-node.sh script to allow node projects...

https://gerrit.wikimedia.org/r/606241

Change 606253 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] WIP: Codehealth pipeline NodeJS configuration

https://gerrit.wikimedia.org/r/606253

@Jdforrester-WMF I started working on a patch for this but have realized that we might need to integrate NodeJS projects (or at least mediawiki/services/push-notifications) into the codehealth pipeline in a different way than we've done for Java and MediaWiki extensions + core.

push-notifications is using a pipeline, so should we be adding configuration to its .pipeline/config.yaml and .pipeline/blubber.yaml files to handle reporting to SonarQube as part of the pipeline build? Or should we go with the path of defining a project + job-templates as the current patch is proposing to do?

@kostajh thanks for the initial feedback and work on this, I have been testing sonar cloud integration with Push-Notification-Service [1] and have reached a problem with code coverage.

The problem is that lcov.info file contains absolute paths [2] when generated, is that something that should be a concern when the pipeline is enabled for node services? Here is the output when running the sonar scanning container:

INFO: Analysing [/workspace/src/coverage/lcov.info]
WARN: Could not resolve 15 file paths in [/workspace/src/coverage/lcov.info]
WARN: First unresolved path: /home/mateus/workspace/wikimedia/infrastructure/push-notifications/src/app.ts (Run in DEBUG mode to get full list of unresolved paths)

[1] https://sonarcloud.io/dashboard?id=thesocialdev_mediawiki-services-push-notifications
[2] https://github.com/coverlet-coverage/coverlet/issues/263

@MSantos yeah, this is annoying, so what we need to do in the mediawiki-services.yaml config as a step just before executing sonar-scanner is something similar to what we do for MediaWiki extensions:

- shell: |
    if [ -f src/extensions/"$EXT_NAME"/coverage/lcov.info ]; then
      sed -ie "s#/src/#/workspace/src/extensions/$EXT_NAME/#g" src/extensions/"$EXT_NAME"/coverage/lcov.info
    fi

This is in line ~636 from mediawiki.yaml in the integration/config repo.

Change 606241 merged by jenkins-bot:
[integration/config@master] dockerfiles: [java8-sonar-scanner] Add a generic run.sh script

https://gerrit.wikimedia.org/r/606241

Mentioned in SAL (#wikimedia-releng) [2020-06-18T16:44:32Z] <James_F> Docker: Publishing java8-sonar-scanner:0.6.1 for T240989

Change 606253 merged by jenkins-bot:
[integration/config@master] jjb: Codehealth pipeline NodeJS configuration

https://gerrit.wikimedia.org/r/606253

Mentioned in SAL (#wikimedia-releng) [2020-06-18T17:36:40Z] <James_F> Zuul: [mediawiki/services/push-notifications] Enable new node codehealth job T240989 T251441

Change 606495 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/config@master] jjb: Use run.sh entrypoint for NodeJS codehealth job

https://gerrit.wikimedia.org/r/606495

In T240989#6233275, @kostajh wrote:

@Jdforrester-WMF I started working on a patch for this but have realized that we might need to integrate NodeJS projects (or at least mediawiki/services/push-notifications) into the codehealth pipeline in a different way than we've done for Java and MediaWiki extensions + core.

push-notifications is using a pipeline, so should we be adding configuration to its .pipeline/config.yaml and .pipeline/blubber.yaml files to handle reporting to SonarQube as part of the pipeline build? Or should we go with the path of defining a project + job-templates as the current patch is proposing to do?

Since are you are already using the pipeline, and because that is different from how the other projects are setup, I think we should try to implement the codehealth check in the pipeline. Or at least consider doing it that way.

You'd want to do something similar to what the above patch proposes (generate test coverage, then trigger a call to sonar-scanner which pushes the analysis to SonarQube, and make sure the credentials are available via an environment variable). The only problem I see is how to use the sonar-scanner CLI from within the pipeline context. If I understand what's happening in blubber.yaml, all the operations are happening in the context of a single docker image, so you'd need a script that would download the sonar-scanner.zip (https://docs.sonarqube.org/latest/analysis/scan/sonarscanner/) and then install and run it; I checked and the sonar-scanner binary should run on docker-registry.wikimedia.org/buster-nodejs10-devel.

In T240989#6244413, @kostajh wrote:

[...] I checked and the sonar-scanner binary should run on docker-registry.wikimedia.org/buster-nodejs10-devel.

I'm getting an error with missing Java environments:

Could not find 'java' executable in JAVA_HOME or PATH.

@kostajh how did you get it working?

Update: I think I got it workable, but I need help with the environment variables:

• See pipeline failure: https://integration.wikimedia.org/ci/job/typescript-service-pipeline-test/141/console
• Patch: https://gerrit.wikimedia.org/r/c/mediawiki/services/push-notifications/+/604830

Change 604830 had a related patch set uploaded (by MSantos; owner: MSantos):
[mediawiki/services/push-notifications@master] WIP: Enable code-health pipeline

https://gerrit.wikimedia.org/r/604830

Change 606495 abandoned by Kosta Harlan:
jjb: [nodejs-codehealth-*] Use run.sh entrypoint

Reason:
We're trying a different approach via the pipeline

https://gerrit.wikimedia.org/r/606495

In T240989#6244413, @kostajh wrote:

In T240989#6233275, @kostajh wrote:

@Jdforrester-WMF I started working on a patch for this but have realized that we might need to integrate NodeJS projects (or at least mediawiki/services/push-notifications) into the codehealth pipeline in a different way than we've done for Java and MediaWiki extensions + core.

push-notifications is using a pipeline, so should we be adding configuration to its .pipeline/config.yaml and .pipeline/blubber.yaml files to handle reporting to SonarQube as part of the pipeline build? Or should we go with the path of defining a project + job-templates as the current patch is proposing to do?

Since are you are already using the pipeline, and because that is different from how the other projects are setup, I think we should try to implement the codehealth check in the pipeline. Or at least consider doing it that way.

You'd want to do something similar to what the above patch proposes (generate test coverage, then trigger a call to sonar-scanner which pushes the analysis to SonarQube, and make sure the credentials are available via an environment variable). The only problem I see is how to use the sonar-scanner CLI from within the pipeline context. If I understand what's happening in blubber.yaml, all the operations are happening in the context of a single docker image, so you'd need a script that would download the sonar-scanner.zip (https://docs.sonarqube.org/latest/analysis/scan/sonarscanner/) and then install and run it; I checked and the sonar-scanner binary should run on docker-registry.wikimedia.org/buster-nodejs10-devel.

It turns out, this approach is not feasible, see https://gerrit.wikimedia.org/r/c/mediawiki/services/push-notifications/+/604830/11#message-0b3666581322f87fe7baa15ecdce3b5ee54f23a5

We should get back to the previous approach

@MSantos I added a new task to add passing environment vars to pipelinelib: T257526: Pipelinelib: support passing environment variables to docker images

Change 604830 merged by jenkins-bot:
[mediawiki/services/push-notifications@master] Enable code-health pipeline

https://gerrit.wikimedia.org/r/604830