Move MediaWiki extension PHPUnit coverage jobs to docker + quibble
Closed, ResolvedPublic

Description

Quibble is the future!

Job
mwext-phpunit-coverage-patch DONE
mwext-phpunit-coverage-publish DONE
mediawiki-phpunit-coverage-patch DONE
mediawiki-core-code-coverage DONE
mediawiki-phpunit-coverage-patch DONE

The jobs run on Nodepool Jessie instances with php7.0.

There are some tricks to be ported:

  • composer require mediawiki/phpunit-patch-coverage 0.0.9

That is done either in MW_INSTALL_PATH or at the root of the extension triggering the patch. mediawiki/phpunit-patch-coverage looks at the files changed in the patchset and does a comparison.

Unfortunately:

The current working directory must be your git repository

  • git checkout composer.json
  1. FIXME This is terrible, but composer require creates a dirty diff.

I am not sure whether we care?

  • tweak tests/phpunit/suite.xml

The jjb macro prepare-mwext-coverage mangles suites.xml to adjust the coverage path filters.

/srv/deployment/integration/slave-scripts/bin/suite-edit.py $MW_INSTALL_PATH/tests/phpunit/suite.xml --cover-extension "$EXT_NAME"
  • new command to be added to Quibble:
./vendor/bin/phpunit-patch-coverage check --command \
    "php7.0 -d zend_extension=xdebug.so $MW_INSTALL_PATH/tests/phpunit/phpunit.php" \
    --html $WORKSPACE/src/coverage.html

Can be a new phpunit-coverage stage. That should be straightforward.

  • archive src/coverage.html

The publish job is slightly easier since it does not need mediawiki/phpunit-patch-coverage :]

Legoktm created this task.May 29 2018, 9:04 PM
Vvjjkkii renamed this task from Move MediaWiki extension PHPUnit coverage jobs to docker + quibble to 50baaaaaaa.Jul 1 2018, 1:07 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot renamed this task from 50baaaaaaa to Move MediaWiki extension PHPUnit coverage jobs to docker + quibble.

@Legoktm can't we add mediawiki/phpunit-patch-coverage 0.0.9 to mediawiki/core require-dev?

Change 446494 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: add phpunit-patch-coverage for Quibble

https://gerrit.wikimedia.org/r/446494

Having phpunit-patch-coverage in the Quibble Docker containers mean that locally people would have to install it manually. That might not be ideal.

The alternatives are:

  • have Quibble install it as part of the flow. The drawback is that we will need to rebuild the CI docker images whenever we need to change the version.
  • add it as a require-dev dependency in mediawiki/core so that any developer would have the package already. The Quibble CI jobs would install it as part of the common flow.

I think it is easier overall to just add it to require-dev. Developers would have the utility locally on their machine, bumping the version is straightforward and does not require to rebuild the Docker images + bump the version in JJB. A drawback is that we end up installing it with all its dependencies for any build, but it is rather small (7MBytes).

hashar moved this task from Backlog to In progress on the Quibble board.Jul 23 2018, 2:53 PM

Change 447447 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Move run/skip in their own arguments group

https://gerrit.wikimedia.org/r/447447

Change 447448 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Add --skip=all to skip all tests

https://gerrit.wikimedia.org/r/447448

Change 447449 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Run any shell command inside Quibble test env

https://gerrit.wikimedia.org/r/447449

Change 447447 merged by jenkins-bot:
[integration/quibble@master] Move run/skip in their own arguments group

https://gerrit.wikimedia.org/r/447447

Change 447448 merged by jenkins-bot:
[integration/quibble@master] Add --skip=all to skip all tests

https://gerrit.wikimedia.org/r/447448

Change 447449 merged by jenkins-bot:
[integration/quibble@master] Run any command inside Quibble test env

https://gerrit.wikimedia.org/r/447449

Change 448477 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: quibble 0.0.21 + rebuild images

https://gerrit.wikimedia.org/r/448477

Change 448477 merged by jenkins-bot:
[integration/config@master] docker: quibble 0.0.21 + rebuild images

https://gerrit.wikimedia.org/r/448477

Change 448549 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Extension coverage jobs on Docker

https://gerrit.wikimedia.org/r/448549

Change 446494 merged by jenkins-bot:
[integration/config@master] docker: add patch coverage utilities to Quibble

https://gerrit.wikimedia.org/r/446494

Mentioned in SAL (#wikimedia-releng) [2018-07-27T14:58:56Z] <hashar> building releng/quibble-stretch:0.0.21-1 container | T195918

Change 448577 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: add xdebug to Quibble stretch images

https://gerrit.wikimedia.org/r/448577

Change 448577 merged by Hashar:
[integration/config@master] docker: add xdebug to Quibble stretch images

https://gerrit.wikimedia.org/r/448577

hashar claimed this task.Jul 27 2018, 4:09 PM

I focused first on mwext-phpunit-coverage-patch-docker which worked fine on my local machine after a few iteration. I broke docker-pkg locally / or it is too slow so I went iterating via building images on contint1001. That is too slow though :(

Will look at finishing that up on Monday.

mwext-phpunit-coverage-publish is a bit more tricky since we need a volume to hold the HTML coverage report. I haven't checked the job generated via https://gerrit.wikimedia.org/r/448549

Last, would need to tackle mediawiki-phpunit-coverage-patch which is triggered by Jenkins and not by Zuul. So that is a slightly different workflow

Change 448549 merged by jenkins-bot:
[integration/config@master] Extension coverage jobs on Docker

https://gerrit.wikimedia.org/r/448549

hashar updated the task description. (Show Details)Jul 30 2018, 10:26 AM

Change 449147 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Fix hardcoded sha1 in mwext-phpunit-coverage-patch.sh

https://gerrit.wikimedia.org/r/449147

Change 449147 merged by jenkins-bot:
[integration/config@master] Fix hardcoded sha1 in mwext-phpunit-coverage-patch.sh

https://gerrit.wikimedia.org/r/449147

Change 449150 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] MediaWiki patch coverage job to Quibble/Docker

https://gerrit.wikimedia.org/r/449150

hashar updated the task description. (Show Details)Jul 30 2018, 1:58 PM

Change 449150 merged by jenkins-bot:
[integration/config@master] MediaWiki patch coverage job to Quibble/Docker

https://gerrit.wikimedia.org/r/449150

Left to migrate is the daily job mediawiki-core-code-coverage which is triggered by Jenkins, not Zuul.

Change 449239 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] MediaWiki patch coverage job to Quibble/Docker

https://gerrit.wikimedia.org/r/449239

Change 449239 merged by jenkins-bot:
[integration/config@master] MediaWiki patch coverage job to Quibble/Docker

https://gerrit.wikimedia.org/r/449239

greg triaged this task as Normal priority.Jul 30 2018, 4:32 PM

Change 449272 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: drop filter from mediawiki-coverage.sh

https://gerrit.wikimedia.org/r/449272

Change 449273 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Migrate mw coverage job to Docker

https://gerrit.wikimedia.org/r/449273

Change 449272 merged by jenkins-bot:
[integration/config@master] docker: drop filter from mediawiki-coverage.sh

https://gerrit.wikimedia.org/r/449272

Mentioned in SAL (#wikimedia-releng) [2018-07-30T19:44:07Z] <hasharAway> renaming Jenkins job "mediawiki-core-code-coverage" to "mediawiki-core-code-coverage-docker" to keep history. It is migrated by https://gerrit.wikimedia.org/r/#/c/integration/config/+/449273/ | T195918

hashar updated the task description. (Show Details)Jul 30 2018, 8:18 PM
hashar updated the task description. (Show Details)Jul 31 2018, 6:24 AM

Change 449273 merged by jenkins-bot:
[integration/config@master] Migrate mw coverage job to Docker

https://gerrit.wikimedia.org/r/449273

hashar updated the task description. (Show Details)Jul 31 2018, 7:29 AM
hashar closed this task as Resolved.