Page MenuHomePhabricator

Investigate potential issues with pcov in phpunit-coverage-php74-docker-publish
Closed, ResolvedPublic

Description

Until earlier today, the job was broken because xdebug was not configured, see example for tools/codesniffer:

+ exec docker run --entrypoint=php --workdir=/src --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/src:/src --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/cache:/cache --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/log:/log --security-opt seccomp=unconfined --init --rm --label jenkins.job=phpunit-coverage-php74-docker-publish --label jenkins.build=14 --env-file /dev/fd/63 docker-registry.wikimedia.org/releng/composer-php74:0.2.0-s9 -d zend_extension=xdebug.so ./vendor/bin/phpunit --coverage-clover /log/coverage/clover.xml --coverage-html /log/coverage
++ /usr/bin/env
++ egrep -v '^(HOME|SHELL|PATH|LOGNAME|MAIL)='
PHPUnit 8.5.30 #StandWithUkraine

Error:         XDEBUG_MODE=coverage or xdebug.mode=coverage has to be set

........................................................          56 / 56 (100%)

Time: 56 ms, Memory: 6.00 MB

I don't know for how long it's been broken, but https://doc.wikimedia.org/cover/mediawiki-tools-codesniffer/ had this at the bottom:

Generated by php-code-coverage 7.0.15 using PHP 7.2.34-18+0~20210223.60+debian10~1.gbpb21322+wmf3 with Xdebug 2.7.0 and PHPUnit 8.5.27 at Thu Jul 14 1:38:25 UTC 2022.

So probably a long time.

Now the job uses pcov, which we did as part of T269489. However, there are some differences in the output:

+ exec docker run --entrypoint=php --workdir=/src --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/src:/src --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/cache:/cache --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/log:/log --security-opt seccomp=unconfined --init --rm --label jenkins.job=phpunit-coverage-php74-docker-publish --label jenkins.build=24 --env-file /dev/fd/63 docker-registry.wikimedia.org/releng/composer-php74:0.2.0-s9 -d extension=pcov.so -d pcov.enable=1 ./vendor/bin/phpunit --coverage-clover /log/coverage/clover.xml --coverage-html /log/coverage
++ /usr/bin/env
++ egrep -v '^(HOME|SHELL|PATH|LOGNAME|MAIL)='
PHPUnit 8.5.30 #StandWithUkraine


Error:         Package phpunit/php-invoker is required for enforcing time limits
...............................................................  63 / 106 ( 59%)
...........................................                     106 / 106 (100%)

Time: 4.34 seconds, Memory: 24.00 MB

Most notably:

  • Number of tests went from 56 to 106.
  • PHPUnit now complains about missing php-invoker. This is expected if the PHPUnit config enables time limits but the package is not installed.

Additionally, the job is failing for mediawiki-libs-Minify:

+ exec docker run --entrypoint=php --workdir=/src --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/src:/src --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/cache:/cache --volume /srv/jenkins/workspace/phpunit-coverage-php74-docker-publish/log:/log --security-opt seccomp=unconfined --init --rm --label jenkins.job=phpunit-coverage-php74-docker-publish --label jenkins.build=25 --env-file /dev/fd/63 docker-registry.wikimedia.org/releng/composer-php74:0.2.0-s9 -d extension=pcov.so -d pcov.enable=1 ./vendor/bin/phpunit --coverage-clover /log/coverage/clover.xml --coverage-html /log/coverage
++ /usr/bin/env
++ egrep -v '^(HOME|SHELL|PATH|LOGNAME|MAIL)='
PHPUnit 8.5.30 #StandWithUkraine

...............................................................  63 / 842 (  7%)
............................................................... 126 / 842 ( 14%)
...........................E................................... 189 / 842 ( 22%)
............................................................... 252 / 842 ( 29%)
............................................................... 315 / 842 ( 37%)
............................................................... 378 / 842 ( 44%)
............................................................... 441 / 842 ( 52%)
............................................................... 504 / 842 ( 59%)
............................................................... 567 / 842 ( 67%)
............................................................... 630 / 842 ( 74%)
............................................................... 693 / 842 ( 82%)
............................................................... 756 / 842 ( 89%)
............................................................... 819 / 842 ( 97%)
.......................                                         842 / 842 (100%)

Time: 424 ms, Memory: 18.00 MB

There was 1 error:

1) CliTest::testRunJsFile
PHPUnit\Framework\Exception: PHP Fatal error:  Uncaught SebastianBergmann\CodeCoverage\RuntimeException: No code coverage driver available in /src/vendor/phpunit/php-code-coverage/src/CodeCoverage.php:907
Stack trace:
#0 /src/vendor/phpunit/php-code-coverage/src/CodeCoverage.php(147): SebastianBergmann\CodeCoverage\CodeCoverage->selectDriver()
#1 Standard input code(302): SebastianBergmann\CodeCoverage\CodeCoverage->__construct()
#2 Standard input code(512): __phpunit_run_isolated_test()
#3 {main}
  thrown in /src/vendor/phpunit/php-code-coverage/src/CodeCoverage.php on line 907
Fatal error: Uncaught SebastianBergmann\CodeCoverage\RuntimeException: No code coverage driver available in /src/vendor/phpunit/php-code-coverage/src/CodeCoverage.php:907
Stack trace:
#0 /src/vendor/phpunit/php-code-coverage/src/CodeCoverage.php(147): SebastianBergmann\CodeCoverage\CodeCoverage->selectDriver()
#1 Standard input code(302): SebastianBergmann\CodeCoverage\CodeCoverage->__construct()
#2 Standard input code(512): __phpunit_run_isolated_test()
#3 {main}
  thrown in /src/vendor/phpunit/php-code-coverage/src/CodeCoverage.php on line 907

phpvfscomposer:///src/vendor/phpunit/phpunit/phpunit:97

This task has the goal of investigating these failures and determing what caused them, and how they should be fixed.

Event Timeline

First question answered: the first output above is not from codesniffer, but from php-session-serializer. So obviously they run different tests etc.

Second question answered: codesniffer's phpunit.xml.dist had

enforceTimeLimit="true"

this was unused, and as noted in the output above, it requires the php-invoker package which is not installed. This is the only repo which still sets this option (I remember it being removed from other places in the past), so we can just unset it now: https://gerrit.wikimedia.org/r/c/mediawiki/tools/codesniffer/+/838871.

Third and last question partly answered: testRunJsFile uses the @runInSeparateProcess to, well, run in a separate process. I suspect that the CLI options for enabling pcov are not being passed through. This seems to be the only test in codesearch using the option, and codesniffer also forbids the annotation. I think the easiest thing would be to refactor the test to avoid the option.

Daimona claimed this task.

Third and last question partly answered: testRunJsFile uses the @runInSeparateProcess to, well, run in a separate process. I suspect that the CLI options for enabling pcov are not being passed through. This seems to be the only test in codesearch using the option, and codesniffer also forbids the annotation. I think the easiest thing would be to refactor the test to avoid the option.

Task for this: T319459. Let's close this one now. New issues can be surfaced as they appear.