Page MenuHomePhabricator

PHPUnit tests do not get run by Jenkins for Gather commits
Closed, ResolvedPublic

Description

Please can they?!

Event Timeline

Jdlrobson raised the priority of this task from to Needs Triage.
Jdlrobson updated the task description. (Show Details)
Jdlrobson moved this task to Needs triage on the Gather board.
Jdlrobson subscribed.
hashar claimed this task.
hashar subscribed.

We run the mediawiki-extensions-hhvm and mediawiki-extensions-zend jobs which are invoking the MediaWiki core extensions testsuite:

php phpunit.php --testsuite extensions

From the test result of a random build, that lists tests SpecialGatherTest and SpecialGatherTest::testCheckRoute. So I guess it works.

hashar triaged this task as Medium priority.Apr 29 2015, 9:54 AM
hashar raised the priority of this task from Medium to High.Apr 29 2015, 10:00 AM
hashar moved this task from Backlog to Archive on the Continuous-Integration-Config board.

Really? We submitted a patch causing tests to fail which didn't get picked up by Jenkins.
We have some api tests that should be running as well - api/GatherTests are those not being picked up?

cc @Yurik who said this wasn't setup.

I have closed the task because some random tests were showing up. Indeed api/GatherTests.php is not run because it is not recognized by the testsuite. The file suffix must be in singular form so the file has to be renamed:

- ./tests/phpunit/api/GatherTests.php
                                ^
+ ./tests/phpunit/api/GatherTest.php

The full explanation:

The tests are registered by Gather using the UnitTestsList hook which has:

$files[] = __DIR__ . '/../tests/phpunit';

The MediaWiki 'extension' testsuite is defined in tests/phpunit/suites/ExtensionsTestSuite.php. When given a directory, the suite lookups for files using the 'Test.php' suffix:

if ( is_dir( $path ) ) {
     $suffixes = array(
         'Test.php',  # Suffix used to name files
     );
     $fileIterator = new File_Iterator_Facade();
     $matchingFiles = $fileIterator->getFilesAsArray( $path, $suffixes );
     $this->addTestFiles( $matchingFiles );

Renaming the file should do it.

Change 209032 had a related patch set uploaded (by Jdlrobson):
Make the tests run in Jenkins

https://gerrit.wikimedia.org/r/209032

Jdlrobson lowered the priority of this task from High to Medium.May 6 2015, 10:00 AM

@Tgr could you work out what's going on here and take over this patch? The tests run locally but not on Jenkins meaning we have no test coverage...

Any updates on this? @Tgr were you able to take a look?

I wasn't, but at a glance the Zend issue is a real bug which depends on PHP engine and version and the other (page order in the listpages API response is different locally and in Jenkins) is maybe a side effect of different phpunit settings wrt database reuse? I would say just kill that one assertion and get the tests running on Jenkins.

Change 209032 merged by jenkins-bot:
Make the tests run in Jenkins

https://gerrit.wikimedia.org/r/209032