Page MenuHomePhabricator

Patches against release branches generate coverage that is published to doc.wikimedia.org
Closed, ResolvedPublic

Description

See https://doc.wikimedia.org/cover-extensions/AbuseFilter/, the footer says:

Generated by php-code-coverage 5.3.2 using PHP 7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1 with Xdebug 2.7.0 and PHPUnit 6.5.14 at Thu Dec 31 11:27:48 UTC 2020.

Which is very much outdated, and shows a very old version of the codebase. I eventually understood that it was generated by r652678, a commit against REL1_31 merged today. I believe these should not generate coverage reports, or at least not mixed up with master reports.

Event Timeline

Daimona triaged this task as High priority.EditedDec 31 2020, 1:48 PM
Daimona added a subscriber: Legoktm.

FTR, the outdated reports are going to be generated for lots of repos because LibUp is currently in the process of upgrading some npm dependencies in REL1_31, see https://gerrit.wikimedia.org/r/q/owner:tools.libraryupgrader%2540tools.wmflabs.org+branch:REL1_31 and T236071

Krinkle added a subscriber: Krinkle.

Indeed, these aren't meant to be enabled for non-master commits. I'm pretty sure this was working correctly at some point, but I can see how that's easily missed during refactors as it doesn't look obviously broken when they run by accident.

The job is named mwext-phpunit-coverage-docker-publish, and the zuul config has:

# Only generate coverage information for master
- name: ^.*-coverage-(docker-)?publish$
  branch: master

So...not sure why this isn't working.

Oh, maybe the earlier mutex rule is overriding that?

# For coverage report jobs, we only want one build at a time.
#
# That is the equivalent of concurrent: false in Jenkins but handled by Zuul
# instead.
- name: mwext-phpunit-coverage-docker-publish
  mutex: mwext-phpunit-coverage-docker-publish

Change 652923 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[integration/config@master] Only run mwext coverage jobs on master patches, really

https://gerrit.wikimedia.org/r/652923

Change 652923 merged by jenkins-bot:
[integration/config@master] Only run mwext coverage jobs on master patches, really

https://gerrit.wikimedia.org/r/652923

Mentioned in SAL (#wikimedia-releng) [2021-01-01T18:26:19Z] <James_F> zuul: Try in a second way to only run mwext coverage jobs on master T270976

Seems Zuul configuration is sensitive to the order in which parameters are applied. Specially applying branch: master via ^.*-coverage-(docker-)?publish$ comes after the job specific rule to set the mutex. To reproduce:

bad.yaml
pipelines:
 - name: coverage
   manager: IndependentPipelineManager
   source: gerrit
   trigger:
       gerrit: []
      
jobs:
    - name: mwext-phpunit-coverage-docker-publish
      mutex: somemutex
    - name: ^.*-coverage-(docker-)?publish$
      branch: master
      
projects: 
    - name: mediawiki/extensions/Foobar
      coverage:
          - mwext-phpunit-coverage-docker-publish

Then:

./utils/zuul-layout-tester.sh zuul-cover.yaml
Configured Pipeline Manager coverage
  Source: <GerritSource connection: gerrit://gerrit>
  Requirements:
  Events:
  Projects:
    mediawiki/extensions/Foobar
      <Job mwext-phpunit-coverage-docker-publish> [mutex: somemutex]
                                                ^^^
                                            [no branch]
...

So looks like the regex jobs (also known as metajobs) have to come first :(

Change 654213 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] zuul: make coverage publish meta job appear first

https://gerrit.wikimedia.org/r/654213

Change 654213 merged by jenkins-bot:
[integration/config@master] zuul: make coverage publish meta job appear first

https://gerrit.wikimedia.org/r/654213

Change 659223 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] zuul: move meta-jobs at the top

https://gerrit.wikimedia.org/r/659223

Change 659223 merged by jenkins-bot:
[integration/config@master] zuul: move meta-jobs at the top

https://gerrit.wikimedia.org/r/659223

Moved all the meta jobs definitions (those having a regex) to the top of zuul/layout.yaml . So we are covered in the future.