Add CodeHealth template to query-builder pipeline
Open, MediumPublic
Actions

Assigned To

None

Authored By

	Michael
	Mar 29 2021, 4:02 PM

Description

CodeHealth is a non-voting job in out integration configuration that runs sonarcube for several of our projects.

It is used by adding a line in the template section of a job, see for example the configuration for Lexeme:

https://github.com/wikimedia/integration-config/blob/master/zuul/layout.yaml#L4589-L4595

The Query Builder configuration was added in https://gerrit.wikimedia.org/r/c/integration/config/+/670898

The code-health template is defined further up in that file:
https://github.com/wikimedia/integration-config/blob/60f01f51f24507521c4c9fbc61f06cb6a1ac0a01/zuul/layout.yaml#L1585-L1589

And more details about how these jobs are run are in their definition: https://github.com/wikimedia/integration-config/blob/60f01f51f24507521c4c9fbc61f06cb6a1ac0a01/jjb/mediawiki.yaml#L647

More general information and tutorials about WMF CI can be found at: https://www.mediawiki.org/wiki/Continuous_integration

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	[DNM] Add node codehealth template and project	integration/config	master	+81 -0

Customize query in gerrit

Related Objects

Mentioned In: T278706: Add branch hosting to Query Builder integration pipeline
Mentioned Here: T265893: Add Link engineering: Deployment Pipeline setup
T266110: Add Link engineering: Codehealth integration

Event Timeline

Michael created this task.Mar 29 2021, 4:02 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 29 2021, 4:02 PM

Michael updated the task description. (Show Details)Mar 29 2021, 4:20 PM

Michael moved this task from Incoming to Ready to pick up on the Wikidata Query Builder board.Mar 31 2021, 9:59 AM

Ladsgroup claimed this task.Apr 6 2021, 10:06 AM

Ladsgroup moved this task from Ready to pick up to Doing on the Wikidata Query Builder board.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptApr 6 2021, 10:06 AM

This is not possible for QB (correct me if I'm missing something obvious). Those jobs are for mwext (mw extensions) and highly tailored for those, like requiring composer, installing core, finding extension dependencies and installing those, paths to find the code, runs phpunit, etc. I couldn't find a general job for codehealth/sonar. I can try to build one but that's not super straightforward.

Ladsgroup added a project: SonarQube Bot.Apr 6 2021, 10:13 PM

Ladsgroup removed Ladsgroup as the assignee of this task.Apr 7 2021, 9:54 AM

Ladsgroup moved this task from Doing to Incoming on the Wikidata Query Builder board.

Ladsgroup subscribed.

This is about restoring some functionality that we had on GitHub to also work on Jenkins. The effort is apparently not as trivial as hoped, but still shouldn't be huge either. Basically we need to copy the existing job (-template) and drop all the mediawiki/php/server funtionality.

This task could be a useful learning for us as a department to reuse for future projects that are hosted on Gerrrit and do not directly integrated into MediaWiki. It would also be a good learning for the individual that implements it towards improving their Skill Matrix score for "WMF Jenkins + Zuul configuration".

Michael mentioned this in T278706: Add branch hosting to Query Builder integration pipeline.Apr 9 2021, 11:25 AM

Michael moved this task from Incoming to Backlog on the Wikidata Query Builder board.Apr 9 2021, 11:28 AM

If I can help, let me know. We have a few non mediawiki jobs in the codehealth pipeline for various languages (Java, python) using a few different configuration set ups.

In T278705#6987282, @kostajh wrote:

If I can help, let me know. We have a few non mediawiki jobs in the codehealth pipeline for various languages (Java, python) using a few different configuration set ups.

Thank you! It shouldn't be hard (™). This project is just a pure static javascript frontend. It needs (as minimum) node10 and npm 6.9.x to install the dependencies and run the tests. We're already using the default node10-test docker image that is on wmf-CI. That works fine (now).

All that is needed is to run the tests not with npm run test but with npm run test:unit -- --coverage and then to trigger the sonar setup. That's it.

It really shouldn't be difficult, probably every other configuration would be more complex. But I hadn't had the peace of mind to look into so far.

If there already exists something like that, that would be great! Otherwise, I think pointers to other simple job/job-template/container-setups for sonar on wmf-CI would also help and be appreciated :)

In T278705#6987349, @Michael wrote:

In T278705#6987282, @kostajh wrote:

If I can help, let me know. We have a few non mediawiki jobs in the codehealth pipeline for various languages (Java, python) using a few different configuration set ups.

Thank you! It shouldn't be hard (™). This project is just a pure static javascript frontend. It needs (as minimum) node10 and npm 6.9.x to install the dependencies and run the tests. We're already using the default node10-test docker image that is on wmf-CI. That works fine (now).

All that is needed is to run the tests not with npm run test but with npm run test:unit -- --coverage and then to trigger the sonar setup. That's it.

It really shouldn't be difficult, probably every other configuration would be more complex. But I hadn't had the peace of mind to look into so far.

If there already exists something like that, that would be great! Otherwise, I think pointers to other simple job/job-template/container-setups for sonar on wmf-CI would also help and be appreciated :)

@Zbyszko would you mind pointing @Michael to one or more of the patches you used for the java / python projects recently?

For a deployment pipeline example (though it looks like you are not using the deployment pipeline), there are some patches linked to T266110: Add Link engineering: Codehealth integration and T265893: Add Link engineering: Deployment Pipeline setup that you could reference.