Page MenuHomePhabricator

Switch MediaWiki coverage job from PHP 5 to PHP 7
Closed, ResolvedPublic

Description

The Jenkins job mediawiki-core-code-coverage produces code coverage for MediaWiki core. It is run on a daily basis and takes a while:

It uses to take almost 4 hours to generate and nowadays a bit less than 2 hours which is still a lot. Build time trend:

The huge drop from 220 minutes to 100 minutes is git lg 3d7e37edd08edd4504bff920cc0b0972917f5795..4a975b8099ee11b15421d03be02206935a8422f1 which contains the refactoring of parser tests. Potentially parser tests are no longer participating in the coverage. Filled as T147779 for investigation.

The job runs on Trusty instances with Zend PHP5.5 and XDebug. PHPUnit relies on PHP_CodeCoverage which apparently supports Zend PHP 7.0 and phpdbg, the combination being an order of magnitude faster.

Event Timeline

hashar created this task.Oct 10 2016, 8:21 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 10 2016, 8:21 AM

Change 314559 had a related patch set uploaded (by Hashar):
Try MediaWiki code coverage on Jessie/php7

https://gerrit.wikimedia.org/r/314559

Change 314563 had a related patch set uploaded (by Hashar):
contint: add phpdbg for code coverage

https://gerrit.wikimedia.org/r/314563

Change 315048 had a related patch set uploaded (by Hashar):
MediaWiki code coverage job now archive all logs

https://gerrit.wikimedia.org/r/315048

Change 315048 merged by jenkins-bot:
MediaWiki code coverage job now archive all logs

https://gerrit.wikimedia.org/r/315048

Krinkle updated the task description. (Show Details)Oct 10 2016, 10:50 PM

Change 314563 merged by Dzahn:
contint: add phpdbg for code coverage

https://gerrit.wikimedia.org/r/314563

hashar lowered the priority of this task from Normal to Low.Nov 23 2016, 1:09 PM
Krinkle added a comment.EditedMar 18 2017, 2:01 AM

@hashar Switching from Trusty/PHP 5.5 to Jessie/PHP 7 (or PHP 5.6) is definitely a good move that will also make things faster.

I thought HHVM might be worth another try. HHVM 3.5.0+ supports the XDebug interface, which php-code-coverage depends on. However, it seems it has still some issues to be resolved https://github.com/sebastianbergmann/php-code-coverage/issues/365 https://github.com/facebook/hhvm/issues/1589.

php-code-coverage 4.0 also brought significant performance improvements, which is available in PHPUnit 5.x and higher. MediaWiki core currently installs PHPUnit 4.x due to PHP 5.5 compat. (PHPUnit 5.x requires PHP 5.6+). We may be able to override that as part of the Jenkins job for code coverage, though.

Krinkle added a comment.EditedMar 18 2017, 2:18 AM

I've got a build in progress at https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-jessie/

  • Build 1: Same job config, but on Jessie/PHP 5.6 instead of Trusty/PHP 5.5
Krinkle added a comment.EditedMar 29 2017, 11:20 PM

I've got a build in progress at https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-jessie/

  • Build 1: Same job config, but on Jessie/PHP 5.6 instead of Trusty/PHP 5.5

These failed due to some unrelated issues with PHPUnit tests in mediawiki-core at the time. Fixed now.

  • Build 4: Completion time: 2 hr 15 min.
  • Build 5 Completion time: 2 hr 18 min.

About 30% faster (1 hour less) than the status quo with PHP 5.5 on Trusty (usually 3 hr+).

Change 345475 had a related patch set uploaded (by Krinkle):
[integration/config@master] Move mw coverage from Trusty (PHP 5.5) to Jessie (PHP 5.6)

https://gerrit.wikimedia.org/r/345475

Change 345475 merged by jenkins-bot:
[integration/config@master] Move mw coverage from Trusty (PHP 5.5) to Jessie (PHP 5.6)

https://gerrit.wikimedia.org/r/345475

Krinkle added a comment.EditedMar 29 2017, 11:34 PM

https://doc.wikimedia.org/cover/mediawiki-core/

  • Before: Generated by PHP_CodeCoverage 2.2.4 using PHP 5.5.9-1ubuntu4.21 and PHPUnit 4.8.31 at Wed Mar 29 15:01:16 UTC 2017.
  • After: Generated by PHP_CodeCoverage 2.2.4 using PHP 5.6.30-0+deb8u1 and PHPUnit 4.8.31 at Wed Mar 29 23:35:30 UTC 2017.

Leaving this task open as the point was to move to PHP7, not PHP 5.6. But this is a small step toward that. There were two issues with PHP 7 when we last tried:

  • Some tests didn't pass. (Fixed I think?)
  • PHPUnit 4.8 supports PHP 5.3-5.6, but not PHP 5.7 or PHP 7.x. (This was fixed in PHPUnit 4.8.21).

Previously, using PHP 7 meant upgrading to PHPUnit 5 or 6 - but we couldn't (and can't) do that since mediawiki supports PHP 5.5; which is not supported by PHPUnit 5.

Krinkle added a comment.EditedMar 31 2017, 4:50 AM

Jessie/PHP 5.6 instead of Trusty/PHP 5.5

I've applied this experimental change from mediawiki-core-code-coverage-jessie to the main "mediawiki-core-code-coverage" job.

Change 345947 had a related patch set uploaded (by Krinkle):
[mediawiki/core@master] changes: Remove assertTag from RCCacheEntryFactoryTest

https://gerrit.wikimedia.org/r/345947

Seb35 added a subscriber: Seb35.Apr 10 2017, 12:04 PM

Change 314559 abandoned by Hashar:
(WIP) Try MediaWiki code coverage on Jessie/php7 (WIP)

Reason:
Timo did some work on the coverage front. Eg Move mw coverage from Trusty (PHP 5.5) to Jessie (PHP 5.6)

https://gerrit.wikimedia.org/r/345475

https://gerrit.wikimedia.org/r/314559

Krinkle renamed this task from Switch MediaWiki coverage job from Trusty/Zend PHP 5.5 to Jessie/Zend PHP 7.0 to Switch MediaWiki coverage job from PHP 5 to PHP 7.Apr 14 2017, 9:31 PM
Legoktm added a subscriber: Legoktm.

Copying my comment from T147781#3874487:

The coverage result is a bit different though, 21.3% (44784/210421 elements) on PHP 5 versus 20.5% (38354/187470 elements) on PHP 7. But it's also nearly twice as fast, which is probably worth pursuing further.

@Legoktm Now that we have the mediawiki-core-code-coverage-php7 job and it's passing as well as the mediawiki-core-code-coverage job, would it make sense to remove the php5-based job and just have the php7 one going forward? That would unblock the chain of parent tasks.

Krinkle raised the priority of this task from Low to High.Apr 10 2018, 1:48 PM

Yeah, once T191863 is fixed, we can drop the php5 one, and rename the php7 one to the standard name.

Change 426377 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[integration/config@master] Run MediaWiki coverage job with PHP 7

https://gerrit.wikimedia.org/r/426377

Change 426377 merged by jenkins-bot:
[integration/config@master] Run MediaWiki coverage job with PHP 7

https://gerrit.wikimedia.org/r/426377

This is still not fixed fully, because the whole install process uses PHP 5, only the last step of running the tests uses PHP 7.

Change 427030 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[integration/config@master] Set $PHP_BIN in MediaWiki coverage job to force everything onto PHP 7

https://gerrit.wikimedia.org/r/427030

Change 427030 merged by jenkins-bot:
[integration/config@master] Set $PHP_BIN in MediaWiki coverage job to force everything onto PHP 7

https://gerrit.wikimedia.org/r/427030

Legoktm closed this task as Resolved.Apr 17 2018, 12:35 AM
Legoktm claimed this task.

Thanks to @Krinkle for talking through it on IRC and pointing out how easy it was to set a global environment variable for a job. :-)

Give or take some varnish cache, https://doc.wikimedia.org/cover/mediawiki-core/index.html now shows the PHPUnit 6 generated coverage report. Notably functions/classes that are marked as @deprecated no longer automatically get 100% coverage.

Very well done! \O/