| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | hashar | T232706 mwcore-phpunit-coverage-master times out after 5 hours | |||
| Open | None | T225730 Reduce runtime of MW shared gate Jenkins jobs to 5 min | |||
| Resolved | Jdforrester-WMF | T280064 Post-merge builds failing with "Command 'mwext-phpunit-coverage' returned non-zero exit status 1" | |||
| Resolved | Jdforrester-WMF | T280167 Tweak pcov ini settings | |||
| Resolved | Daimona | T279423 Enable coverage reports for taint-check | |||
| Resolved | Daimona | T269489 composer-package-phpXX-docker jobs run with xdebug enabled | |||
| Resolved | Daimona | T234020 Switch mediawiki code coverage from xdebug to pcov | |||
| Resolved | Daimona | T280170 Add pcov to composer images | |||
| Open | None | T243847 Add pcov PHP extension to wikimedia apt (and upgrade from 1.0.6-4+wmf1~buster1 to 1.0.11) so it can be used in Wikimedia CI |
Event Timeline
Honestly, I have no idea what the hack for sury is :) Anyway, I guess it would be fine, as long as we can pull an up-to-date version of pcov.
Sorry for being gnomic! :-)
For sury-php, we just inject the personal nightly PHP packages that sury makes before they get properly packaged for debian, and so can use versions which aren't approved for each Debian release: https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/master/dockerfiles/sury-php/Dockerfile.template
Gotcha, thanks! I'll leave that up for you to decide, as I'm not qualified for that. My only thought is that, as long as the solution is reusable, it's fine. I'm specifically thinking about the possible use case in vagrant (somewhat related to T244716).
Alternatively, can we pull pcov from git and compile as per https://github.com/krakjoe/pcov/blob/develop/INSTALL.md#installation ?
The incentive was to benefit from speed improvement. We got Xdebug main slowness identified and solved T234418. phpcov coverage is also slightly different.
While that might have been a slowness cause, it's still insufficient. First of all, we're using xdebug 2.7.*, but 2.9.0 claims a 250% speed up compared to 2.7.0. At least, this is something that could be explored. Second, pcov is still way faster than xdebug 2.9.*, based on local tests; see T234020#5826901 for an example. If the improvement is really that big, it would be foolish not to take advantage of a relatively easy change like switching the coverage driver.
IMHO, we cannot decline this task because "xdebug is already faster" without knowing first how much pcov can be faster than xdebug.
phpcov coverage is also slightly different.
It is, but is that a problem? As I wrote in T234020#5826901, the only notable differences are that pcov ignores lines with a global declaration (i.e. global $wgFoo), and the first line of switch expressions (i.e. switch ( $foo )). Is it bad not to have these lines show up as coverable? IMHO, no. Especially if that comes with a noticeable performance improvement.
Memo (for myself or whoever is interested): xdebug 3 should supposedly bring massive performance improvements. Copying from the release notes for 3.0.0beta1:
This made it possible to massively increase performance in many of Xdebug's sub systems as Xdebug is now much more conservative in which hooks are enabled.
Configuration changes, massive performance improvements, and PHP 8 support are the primary features in Xdebug 3
Once a stable is version is released, we should check its performance compared to pcov. If it turns out to be similar or better, we might consider staying with xdebug, as long as we upgrade it to version 3. Note, IMHO it's still subpar to use an "all-in-one" debugger for coverage (because dedicated coverage drivers like pcov or phpdbg should in principle be faster), but I guess upgrading xdebug would be easier, thus acceptable (at least temporarily).
This is actually still relevant. In particular, given T293851, it would be great to have pcov in wm apt, assuming that we're going to switch coverage jobs to PHP 7.4 at some point.
Mentioned in SAL (#wikimedia-operations) [2021-11-10T19:23:10Z] <legoktm> uploaded php-pcov_1.0.6-4+wmf1~buster1_amd64.changes to apt.wm.o (T243847)
Sorry for reopening this; would it be possible to also add a PHP 7.2 version of pcov, please? Context: I'd like to use it for https://gerrit.wikimedia.org/r/c/integration/config/+/694621.
pcov got build and uploaded to component/php74. Meanwhile upstream released a few bug fixes ( https://pecl.php.net/package-changelog.php?package=pcov ):
1.0.11 - Fix anonymous function coverage 1.0.10 - Use zend_new_interned_string instead of hand-rolling - Use ZEND_MOD_END to fix build warning 1.0.9 - Fix #67 huge memory consumption to generate coverage 1.0.8 - Fix Windows build 1.0.7 - Fix #46 Segmentation fault on 7.4 on macOS with anonymous class - Fix #59 "malloc_consolidate(): invalid chunk size" with PHP 8.1
The invalid chunk size with php 8.1 doesn't affect CI since our images use sury.org which ships pcov 1.0.11 already. But 1.0.9 fix huge memory consumption to generate coverage sounds terrible and would affect our php 7.4 image. The bug https://github.com/krakjoe/pcov/issues/67
We thus need:
- update of our php7.4-pcov package from 1.0.6 to 1.0.11.
- a php7.2 based build of 1.0.11 to solve T243847 which will also ensure all our CI images use the same set of packages.