Change 324186 had a related patch set uploaded (by Hashar):
contint: install php5-xsl package

https://gerrit.wikimedia.org/r/324186

Change 324186 merged by Giuseppe Lavagetto:
contint: install php5-xsl package

https://gerrit.wikimedia.org/r/324186

Mentioned in SAL (#wikimedia-releng) [2016-11-29T14:24:27Z] <hashar> Refreshing Nodepool Trusty snapshot to get php5-xsl installed T151879

Mentioned in SAL (#wikimedia-releng) [2016-11-29T14:30:01Z] <hashar> Image ci-trusty-wikimedia-1480429423 in wmflabs-eqiad is ready T151879

Change 324197 had a related patch set uploaded (by Hashar):
Add phpunit/phpunit 4.8.x to require-dev

https://gerrit.wikimedia.org/r/324197

Thanks hashar for the setup. I will check in details how I can improve the various reports.

Some first things:

• I didn’t try with HHVM since I don’t have it installed but I will install it and test with it locally
• I test often locally with phpdbg 7.0 because it is quicker than php 7.0 and a lot quicker than PHP 5
• I have a special "script" in composer.json to launch PHPUnit directly, but in this case the class MediaWikiTestCase is expectly unknown so I replaced just before with the standard PHPUnit class PHPUnit_Framework_TestCase; it’s quite hacky -- I just realised I could define an empty class MediaWikiTestCase extending PHPUnit_Framework_TestCase in the case PHPUnit is called directly, it will be better (running PHPUnit alone is a lot quicker than MediaWiki’s wrapper, mostly loading time)

For the failed test MultiversionInstallationTest::testDeploymedVersions3, I’m pretty sure it’s either an issue with HHVM. In the function MediaWikiFarm::readFile, when running this test, the code

if( $filename == 'testdeploymentsfarmversions.php' ) var_dump(file_get_contents($prefixedFile));
$array = include $prefixedFile;
if( $filename == 'testdeploymentsfarmversions.php' ) var_dump($array);

returns

string(61) "<?php

return array(
	'atestdeploymentsfarm' => 'vstub2',
);
"
array(1) {
  ["atestdeploymentsfarm"]=>
  string(5) "vstub"
}

The same code executed with PHP 5.6.27 and PHP 7.0.12 returns the correct result. In my case, I have HHVM 3.15.3 (Compiler: tags/HHVM-3.15.3-0-g965cd8d30670726d88720412c3384225a2da011d)

This specific test depends on previous tests. When I don’t execute the first test (with a return as first instruction), the result of the test 3 is correct. Between the two tests, the content of the the file is changed, and it seems HHVM reuses the result from the first read.

I first thought about some internal cache somewhere in HHVM keeping the result of previous file reads (PHP has such mechanism for file metadata/clearstatcache), so I tried to sleep 10 seconds in the beginning of the 3rd test (no improvement), touch the file (no improvement), delete the file (no improvement), clearstatcache (no improvement). So I tend to say it’s a HHVM bug, where some pointer somewhere is reused instead of being deleted and recreated.

Change 324197 merged by Seb35:
Add phpunit/phpunit 4.8.x to require-dev

https://gerrit.wikimedia.org/r/324197

Change 324693 had a related patch set uploaded (by Seb35):
Improve testing

https://gerrit.wikimedia.org/r/324693

For the HHVM thing, it’s the issue #4797. Smarty implemented a workaround.

For now I will just skip this test or assertion with HHVM and will fix it later, partly because I think it would not occur in a real environment where the PHP file would be modified independantly from the PHP execution, so probably HHVM would check the modifications in these cases.

Change 324693 merged by jenkins-bot:
Improve testing infrastructure

https://gerrit.wikimedia.org/r/324693

Impressive ! Merci @Seb35 :}

@hashar Merci pour la mise en place et les conseils ! :]