Page MenuHomePhabricator

Exclude certain jobs from early feedback
Closed, ResolvedPublic

Description

There are some jobs where it doesn't make sense to provide an alert of a failure, because they are non voting. For example, mwext-phpunit patch coverage doesn't need a notification:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/889642/31#message-7c9b65493f93ef9f4c260472090346cab394dd0a

Ideally, there is a way to detect via an environment variable that a job is non-voting. In theory ZUUL_VOTING should be that variable, but for mwext-phpunit-coverage-patch-docker it is set to 1, so I am not sure what is going on.

An alternative is to craft a list of jobs that should never trigger notifications.

Event Timeline

Change 894213 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[integration/quibble@master] reporting: Include `pipeline` in payload

https://gerrit.wikimedia.org/r/894213

Ideally, there is a way to detect via an environment variable that a job is non-voting. In theory ZUUL_VOTING should be that variable, but for mwext-phpunit-coverage-patch-docker it is set to 1, so I am not sure what is going on.

This happens because mwext-phpunit-coverage-patch-docker actually is a voting job (the coverage pipeline does fail when mwext-phpunit-coverage-patch-docker fails, which is the definition of voting according to Zuul's docs), but Zuul is told to only report this pipeline failure as a comment (without casting a Gerrit vote). This is configured in zuul/layout.yaml (note the lack of verified: -1, which is present for some other pipelines):

failure-message: 'PHP test coverage decreased'
failure:
  gerrit:
    tag: autogenerated:ci-coverage

Unfortunately, both Zuul and Gerrit use the word "voting" with different meaning, which is fairly confusing here.

EarlyWarningBot should only vote for the test, test-prio and gate-and-submit pipelines (including their backport-specific variants). As of now, the bot isn't aware of the pipeline the failure occured in. This can be fixed by including the pipeline name in the payload sent to EarlyWarningBot, and only making the bot to only comment for the pipelines we care about. Arguably, a nicer solution could be to include the consequences of the pipeline in the payload (rather than just the pipeline's name), but I'm not sure how that could be accomplished.

kostajh added a subscriber: Urbanecm.

This will be resolved with the deployment of Quibble 1.5.2. Thank you @Urbanecm!

Change 894213 merged by jenkins-bot:

[integration/quibble@master] reporting: Include `pipeline` in payload

https://gerrit.wikimedia.org/r/894213

Change 894535 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/quibble@master] release: Quibble 1.5.2

https://gerrit.wikimedia.org/r/894535

Change 894535 merged by jenkins-bot:

[integration/quibble@master] release: Quibble 1.5.2

https://gerrit.wikimedia.org/r/894535