MediaWikiFarm Jenkins job failures
Closed, ResolvedPublic

Description

I wanted to Jenkins jobs setup for MediaWikiFarm, but they fail on the dummy change https://gerrit.wikimedia.org/r/#/c/324091/


It uses phpdocumentor which requires the PHP extension xsl, unfortunately it is not available on our Trusty instances and fail the build. We have to add the package php5-xsl


The test MultiversionInstallationTest::testDeploymedVersions3 fails under HHVM though it pass just fine with Zend PHP 5.5:

Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'vstub2'
+'vstub'

extensions/MediaWikiFarm/tests/phpunit/MultiversionInstallationTest.php:372
tests/phpunit/MediaWikiTestCase.php:399
src/maintenance/doMaintenance.php:111

Resolution:
Bug in HHVM #4797 https://github.com/facebook/hhvm/issues/4797. I "skipped" the test for now, will try to find an acceptable resolution for this. The template library Smarty implemented a workaround, basically copying the PHP file with a random name and including it. https://github.com/smarty-php/smarty/commit/f9d9ca0f67c6b63e033a7a8e0d1623d60fc0d1ba


composer test fails to find phpunit:

> composer unit
> sed -i 's/extends MediaWikiTestCase/extends PHPUnit_Framework_TestCase/' tests/phpunit/MediaWikiFarmTestCase.php
> sh: 1: phpunit: not found

Resolution:
Added PHPUnit 4.8 as require-dev dependency


phpdbg is missing:

Error Output: sh: 1: phpdbg: not found

Resolution:
Fallback to php if phpdbg is not here


Other issues are detailed in the commit message.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 29 2016, 2:16 PM

Change 324186 had a related patch set uploaded (by Hashar):
contint: install php5-xsl package

https://gerrit.wikimedia.org/r/324186

Change 324186 merged by Giuseppe Lavagetto:
contint: install php5-xsl package

https://gerrit.wikimedia.org/r/324186

Mentioned in SAL (#wikimedia-releng) [2016-11-29T14:24:27Z] <hashar> Refreshing Nodepool Trusty snapshot to get php5-xsl installed T151879

Mentioned in SAL (#wikimedia-releng) [2016-11-29T14:30:01Z] <hashar> Image ci-trusty-wikimedia-1480429423 in wmflabs-eqiad is ready T151879

Change 324197 had a related patch set uploaded (by Hashar):
Add phpunit/phpunit 4.8.x to require-dev

https://gerrit.wikimedia.org/r/324197

hashar updated the task description. (Show Details)Nov 29 2016, 3:19 PM
Seb35 added a comment.Nov 30 2016, 6:35 PM

Thanks hashar for the setup. I will check in details how I can improve the various reports.

Some first things:

  • I didn’t try with HHVM since I don’t have it installed but I will install it and test with it locally
  • I test often locally with phpdbg 7.0 because it is quicker than php 7.0 and a lot quicker than PHP 5
  • I have a special "script" in composer.json to launch PHPUnit directly, but in this case the class MediaWikiTestCase is expectly unknown so I replaced just before with the standard PHPUnit class PHPUnit_Framework_TestCase; it’s quite hacky -- I just realised I could define an empty class MediaWikiTestCase extending PHPUnit_Framework_TestCase in the case PHPUnit is called directly, it will be better (running PHPUnit alone is a lot quicker than MediaWiki’s wrapper, mostly loading time)
Seb35 added a comment.Dec 1 2016, 9:23 AM

For the failed test MultiversionInstallationTest::testDeploymedVersions3, I’m pretty sure it’s either an issue with HHVM. In the function MediaWikiFarm::readFile, when running this test, the code

if( $filename == 'testdeploymentsfarmversions.php' ) var_dump(file_get_contents($prefixedFile));
$array = include $prefixedFile;
if( $filename == 'testdeploymentsfarmversions.php' ) var_dump($array);

returns

string(61) "<?php

return array(
	'atestdeploymentsfarm' => 'vstub2',
);
"
array(1) {
  ["atestdeploymentsfarm"]=>
  string(5) "vstub"
}

The same code executed with PHP 5.6.27 and PHP 7.0.12 returns the correct result. In my case, I have HHVM 3.15.3 (Compiler: tags/HHVM-3.15.3-0-g965cd8d30670726d88720412c3384225a2da011d)

This specific test depends on previous tests. When I don’t execute the first test (with a return as first instruction), the result of the test 3 is correct. Between the two tests, the content of the the file is changed, and it seems HHVM reuses the result from the first read.

Seb35 added a comment.Dec 1 2016, 9:44 AM

I first thought about some internal cache somewhere in HHVM keeping the result of previous file reads (PHP has such mechanism for file metadata/clearstatcache), so I tried to sleep 10 seconds in the beginning of the 3rd test (no improvement), touch the file (no improvement), delete the file (no improvement), clearstatcache (no improvement). So I tend to say it’s a HHVM bug, where some pointer somewhere is reused instead of being deleted and recreated.

Change 324197 merged by Seb35:
Add phpunit/phpunit 4.8.x to require-dev

https://gerrit.wikimedia.org/r/324197

Change 324693 had a related patch set uploaded (by Seb35):
Improve testing

https://gerrit.wikimedia.org/r/324693

Seb35 triaged this task as Normal priority.Dec 1 2016, 10:30 AM
Seb35 updated the task description. (Show Details)
Seb35 claimed this task.Dec 1 2016, 11:24 AM
Seb35 added a comment.Dec 1 2016, 12:14 PM

For the HHVM thing, it’s the issue #4797. Smarty implemented a workaround.

For now I will just skip this test or assertion with HHVM and will fix it later, partly because I think it would not occur in a real environment where the PHP file would be modified independantly from the PHP execution, so probably HHVM would check the modifications in these cases.

Change 324693 merged by jenkins-bot:
Improve testing infrastructure

https://gerrit.wikimedia.org/r/324693

Seb35 updated the task description. (Show Details)Dec 5 2016, 3:07 PM
Seb35 closed this task as Resolved.Dec 5 2016, 3:11 PM

Finally MediaWikiFarm now has Jenkins jobs running for each change, and Jenkins vote +2 Verified instead of me doing it on my own judgement.

Impressive ! Merci @Seb35 :}

Seb35 added a comment.Dec 5 2016, 7:44 PM

@hashar Merci pour la mise en place et les conseils ! :]