Page MenuHomePhabricator

Move MediaWiki extension PHPUnit coverage jobs to docker + quibble
Closed, ResolvedPublic

Description

Quibble is the future!

Job
mwext-phpunit-coverage-patch DONE
mwext-phpunit-coverage-publish DONE
mediawiki-phpunit-coverage-patch DONE
mediawiki-core-code-coverage DONE
mediawiki-phpunit-coverage-patch DONE

The jobs run on Nodepool Jessie instances with php7.0.

There are some tricks to be ported:

  • composer require mediawiki/phpunit-patch-coverage 0.0.9

That is done either in MW_INSTALL_PATH or at the root of the extension triggering the patch. mediawiki/phpunit-patch-coverage looks at the files changed in the patchset and does a comparison.

Unfortunately:

The current working directory must be your git repository

  • git checkout composer.json
  1. FIXME This is terrible, but composer require creates a dirty diff.

I am not sure whether we care?

  • tweak tests/phpunit/suite.xml

The jjb macro prepare-mwext-coverage mangles suites.xml to adjust the coverage path filters.

/srv/deployment/integration/slave-scripts/bin/suite-edit.py $MW_INSTALL_PATH/tests/phpunit/suite.xml --cover-extension "$EXT_NAME"
  • new command to be added to Quibble:
./vendor/bin/phpunit-patch-coverage check --command \
    "php7.0 -d zend_extension=xdebug.so $MW_INSTALL_PATH/tests/phpunit/phpunit.php" \
    --html $WORKSPACE/src/coverage.html

Can be a new phpunit-coverage stage. That should be straightforward.

  • archive src/coverage.html

The publish job is slightly easier since it does not need mediawiki/phpunit-patch-coverage :]

Event Timeline

Vvjjkkii renamed this task from Move MediaWiki extension PHPUnit coverage jobs to docker + quibble to 50baaaaaaa.Jul 1 2018, 1:07 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
CommunityTechBot renamed this task from 50baaaaaaa to Move MediaWiki extension PHPUnit coverage jobs to docker + quibble.Jul 2 2018, 3:19 PM
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot updated the task description. (Show Details)

@Legoktm can't we add mediawiki/phpunit-patch-coverage 0.0.9 to mediawiki/core require-dev?

Change 446494 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: add phpunit-patch-coverage for Quibble

https://gerrit.wikimedia.org/r/446494

Having phpunit-patch-coverage in the Quibble Docker containers mean that locally people would have to install it manually. That might not be ideal.

The alternatives are:

  • have Quibble install it as part of the flow. The drawback is that we will need to rebuild the CI docker images whenever we need to change the version.
  • add it as a require-dev dependency in mediawiki/core so that any developer would have the package already. The Quibble CI jobs would install it as part of the common flow.

I think it is easier overall to just add it to require-dev. Developers would have the utility locally on their machine, bumping the version is straightforward and does not require to rebuild the Docker images + bump the version in JJB. A drawback is that we end up installing it with all its dependencies for any build, but it is rather small (7MBytes).

Change 447447 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Move run/skip in their own arguments group

https://gerrit.wikimedia.org/r/447447

Change 447448 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Add --skip=all to skip all tests

https://gerrit.wikimedia.org/r/447448

Change 447449 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/quibble@master] Run any shell command inside Quibble test env

https://gerrit.wikimedia.org/r/447449

Change 447447 merged by jenkins-bot:
[integration/quibble@master] Move run/skip in their own arguments group

https://gerrit.wikimedia.org/r/447447

Change 447448 merged by jenkins-bot:
[integration/quibble@master] Add --skip=all to skip all tests

https://gerrit.wikimedia.org/r/447448

Change 447449 merged by jenkins-bot:
[integration/quibble@master] Run any command inside Quibble test env

https://gerrit.wikimedia.org/r/447449

Change 448477 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: quibble 0.0.21 + rebuild images

https://gerrit.wikimedia.org/r/448477

Change 448477 merged by jenkins-bot:
[integration/config@master] docker: quibble 0.0.21 + rebuild images

https://gerrit.wikimedia.org/r/448477

Change 448549 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Extension coverage jobs on Docker

https://gerrit.wikimedia.org/r/448549

Change 446494 merged by jenkins-bot:
[integration/config@master] docker: add patch coverage utilities to Quibble

https://gerrit.wikimedia.org/r/446494

Mentioned in SAL (#wikimedia-releng) [2018-07-27T14:58:56Z] <hashar> building releng/quibble-stretch:0.0.21-1 container | T195918

Change 448577 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: add xdebug to Quibble stretch images

https://gerrit.wikimedia.org/r/448577

Change 448577 merged by Hashar:
[integration/config@master] docker: add xdebug to Quibble stretch images

https://gerrit.wikimedia.org/r/448577

I focused first on mwext-phpunit-coverage-patch-docker which worked fine on my local machine after a few iteration. I broke docker-pkg locally / or it is too slow so I went iterating via building images on contint1001. That is too slow though :(

Will look at finishing that up on Monday.

mwext-phpunit-coverage-publish is a bit more tricky since we need a volume to hold the HTML coverage report. I haven't checked the job generated via https://gerrit.wikimedia.org/r/448549

Last, would need to tackle mediawiki-phpunit-coverage-patch which is triggered by Jenkins and not by Zuul. So that is a slightly different workflow

Change 448549 merged by jenkins-bot:
[integration/config@master] Extension coverage jobs on Docker

https://gerrit.wikimedia.org/r/448549

Change 449147 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Fix hardcoded sha1 in mwext-phpunit-coverage-patch.sh

https://gerrit.wikimedia.org/r/449147

Change 449147 merged by jenkins-bot:
[integration/config@master] Fix hardcoded sha1 in mwext-phpunit-coverage-patch.sh

https://gerrit.wikimedia.org/r/449147

Change 449150 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] MediaWiki patch coverage job to Quibble/Docker

https://gerrit.wikimedia.org/r/449150

Change 449150 merged by jenkins-bot:
[integration/config@master] MediaWiki patch coverage job to Quibble/Docker

https://gerrit.wikimedia.org/r/449150

Left to migrate is the daily job mediawiki-core-code-coverage which is triggered by Jenkins, not Zuul.

Change 449239 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] MediaWiki patch coverage job to Quibble/Docker

https://gerrit.wikimedia.org/r/449239

Change 449239 merged by jenkins-bot:
[integration/config@master] MediaWiki patch coverage job to Quibble/Docker

https://gerrit.wikimedia.org/r/449239

greg triaged this task as Medium priority.Jul 30 2018, 4:32 PM

Change 449272 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: drop filter from mediawiki-coverage.sh

https://gerrit.wikimedia.org/r/449272

Change 449273 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Migrate mw coverage job to Docker

https://gerrit.wikimedia.org/r/449273

Change 449272 merged by jenkins-bot:
[integration/config@master] docker: drop filter from mediawiki-coverage.sh

https://gerrit.wikimedia.org/r/449272

Mentioned in SAL (#wikimedia-releng) [2018-07-30T19:44:07Z] <hasharAway> renaming Jenkins job "mediawiki-core-code-coverage" to "mediawiki-core-code-coverage-docker" to keep history. It is migrated by https://gerrit.wikimedia.org/r/#/c/integration/config/+/449273/ | T195918

Change 449273 merged by jenkins-bot:
[integration/config@master] Migrate mw coverage job to Docker

https://gerrit.wikimedia.org/r/449273

hashar updated the task description. (Show Details)