Page MenuHomePhabricator
Create Task
Maniphest T50217

Speed up MediaWiki PHPUnit build by running integration tests in parallel
Open, LowPublic
Actions

Description

PHPUnit jobs have been merged into one job. A few years ago we had separate categories that ran concurrently. They were merged for unrelated reasons, but it did slow things down.

Look into software that would help running these in parallel within the job (separate threads/processes)

See also:
T60771: Jenkins: MediaWiki extensions phpunit test should also run mediawiki core tests

Details

Reference
bz48217
ProjectBranchLines +/-Subject
mediawiki/coremaster+20 -17tests: Fix memcached test failure with multiple BagOStuff test classes
integration/quibblemaster+306 -55[WIP] Run PHPUnit tests in parallel
mediawiki/coremaster+2 -2phpunit: parse any --boostrap parameter in getopt() call
mediawiki/coremaster+18 -22phpunit: Use process ID in test DB prefix
mediawiki/coremaster+0 -4phpunit: Remove redundancy from "skins" test and "extensions" suites
mediawiki/coremaster+268 -98tests: Split out DatabaseSqliteUpgradeTest class
mediawiki/coremaster+2 -2tests: Avoid unsafe use of setUpBeforeClass() in ApiFormatXmlTest
mediawiki/coremaster+137 -2tests: split includes/ tests into suites for tools like paratest
mediawiki/coremaster+20 -9[WIP] phpunit: Support splitting core and extension suites
mediawiki/coremaster+0 -0Move ApiQuery* tests under the /query subdirectory
Customize query in gerrit

Related Objects
Search...

Event Timeline

• bzimport raised the priority of this task from to Low.Nov 22 2014, 1:18 AM
• bzimport added a project: Continuous-Integration-Infrastructure.
• bzimport set Reference to bz48217.
• bzimport added a subscriber: Unknown Object (MLST).
Krinkle created this task.May 7 2013, 4:52 PM
Krinkle added a comment.Jul 16 2013, 4:39 PM

https://github.com/verkkokauppacom/parallel-phpunit

hashar added a comment.Oct 10 2013, 7:29 PM

Gist from Sebastian Bergmann, PHPUnit creator:

partition-testsuite.php

Script that (making way too many assumptions) generates an Apache Ant build script to run a PHPUnit test suite in parallel.

https://gist.github.com/sebastianbergmann/4405417

Then assemble the generated JUnit XML files using:

merge-phpunit-xml.php

Script that merges multiple PHPUnit XML logfiles (JUnit XML) into one PHPUnit XML logfile.

https://gist.github.com/sebastianbergmann/4405658

demon removed a subscriber: demon.Dec 16 2014, 6:05 PM
Krinkle renamed this task from Jenkins: Merge seperate mediawiki-core-phpunit-{group} jobs and use parallel-phpunit to Use parallel-phpunit in MediaWIki PHPUnit test jobs.Feb 16 2015, 3:18 AM
Krinkle set Security to None.
Krinkle removed a subscriber: Unknown Object (MLST).
Krinkle moved this task from Untriaged to Next on the Continuous-Integration-Infrastructure board.Mar 2 2015, 2:55 PM
Krinkle added a project: MediaWiki-Core-Tests.
Krinkle renamed this task from Use parallel-phpunit in MediaWIki PHPUnit test jobs to Use parallel-phpunit in MediaWiki PHPUnit test jobs.Mar 3 2015, 4:38 PM
Jdforrester-WMF added a subscriber: Jdforrester-WMF.Mar 3 2015, 4:40 PM
Krinkle mentioned this in rCICF0240dc7b2cd2: Remove mediawiki-phpunit-zend from mediawiki-core 'test' pipeline.Mar 3 2015, 5:19 PM
Krinkle renamed this task from Use parallel-phpunit in MediaWiki PHPUnit test jobs to Speed up MediaWiki PHPUnit build by running tests in parallel.Apr 1 2015, 4:08 AM
Krinkle updated the task description. (Show Details)
Krinkle added a comment.Apr 1 2015, 4:41 AM

This might work for standalone projects without a database, but I imagine in the case of MediaWiki core the bootstrapping has significant overhead. As well as concurrency issues. Neither of the mentioned frameworks supports parallelising by group.

I think our librarisation effort will effectively parallelise our tests by being in separate independently versioned projects that don't need all tests to run at once.

See also T93556: Unit tests using database are slower than needed.

Krinkle moved this task from Next to Untriaged on the Continuous-Integration-Infrastructure board.Apr 7 2015, 1:21 PM
Krinkle moved this task from Untriaged to Backlog on the Continuous-Integration-Infrastructure board.Apr 7 2015, 2:26 PM
hashar removed a project: Continuous-Integration-Infrastructure.Oct 6 2015, 11:43 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 6 2015, 11:43 AM
Krinkle added a project: Continuous-Integration-Config.Oct 6 2015, 9:15 PM
Ricordisamoa added a subscriber: Ricordisamoa.Jan 4 2016, 4:17 AM
Legoktm added a subscriber: Legoktm.Feb 15 2016, 9:11 AM

For MediaWiki core, all parsertests now run in a separate job from other phpunit tests, roughly cutting the time in half.

Krinkle closed this task as Declined.Jan 13 2017, 12:24 AM
Krinkle added subscribers: tstarling, daniel.

Thanks to the work of @tstarling and @daniel recently, our Parser and Database-related tests are now much faster. It's also become easier to run locally and run only certain subsets of the tests. Considering the full test suite is under 4-5 minutes for Jenkins, and other jobs take longer, I don't think it's worth the complexity to try and run tests in parallel.

kostajh reopened this task as Open.Jun 17 2019, 5:54 PM
kostajh added a subscriber: kostajh.

I'd like to revisit this as an experiment using [fastest](https://github.com/liuggio/fastest). In my local environment using MediaWiki-Docker-Dev, running the full test suite takes 49 minutes. With fastest, it takes 39 minutes:

root@68e6b761fef5:/var/www/mediawiki# find tests/phpunit extensions/**/tests/phpunit skins/**/tests/phpunit -name "*Test.php" | ./vendor/liuggio/fastest/fastest "tests/phpunit/phpunit.php {};"
find: ‘extensions/**/tests/phpunit’: No such file or directory
find: ‘skins/**/tests/phpunit’: No such file or directory
- 662 shuffled test classes into the queue.
- Will be consumed by 4 parallel Processes.




662/662 [============================] 100% 39 mins 6.0 MiB

     8 failures.
[4] tests/phpunit/tests/MediaWikiTestCaseSchema2Test.phpUsing PHP 7.2.14
PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

E

Time: 2.62 seconds, Memory: 30.00MB

There was 1 error:

1) MediaWikiTestCaseSchema2Test::testMediaWikiTestCaseSchemaTestOrder
Error: Class 'MediaWikiTestCaseSchema1Test' not found

/var/www/mediawiki/tests/phpunit/tests/MediaWikiTestCaseSchema2Test.php:25
/var/www/mediawiki/tests/phpunit/MediaWikiTestCase.php:427
/var/www/mediawiki/maintenance/doMaintenance.php:99

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

[2] tests/phpunit/includes/specials/SpecialShortpagesTest.phpUsing PHP 7.2.14
PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

F

Time: 2.52 seconds, Memory: 30.00MB

There was 1 failure:

1) SpecialShortpagesTest::testGetQueryInfoRespectsContentNS with data set #0 (array(0, 6), array(), array(0, 6))
=== Logs generated by test case
[caches] [info] LocalisationCache: using store LCStoreNull {"private":false}
[wfDebug] [debug] SpecialPage::getContext called and $mContext is null. Return RequestContext::getMain(); for sanity {"private":false}
[caches] [info] LocalisationCache: using store LCStoreNull {"private":false}
===
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => 0
-    1 => 6

/var/www/mediawiki/tests/phpunit/includes/specials/SpecialShortpagesTest.php:30
/var/www/mediawiki/tests/phpunit/MediaWikiTestCase.php:427
/var/www/mediawiki/maintenance/doMaintenance.php:99

FAILURES!
Tests: 1, Assertions: 3, Failures: 1.

[1] tests/phpunit/includes/search/SearchResultTest.php
Fatal error: Class 'MediawikiTestCase' not found in /var/www/mediawiki/tests/phpunit/includes/search/SearchResultTest.php on line 3

Call Stack:
    0.0087     406392   1. {main}() /var/www/mediawiki/tests/phpunit/phpunit.php:0
    0.0299     698800   2. require('/var/www/mediawiki/maintenance/doMaintenance.php') /var/www/mediawiki/tests/phpunit/phpunit.php:129
    0.3754   11037136   3. PHPUnitMaintClass->execute() /var/www/mediawiki/maintenance/doMaintenance.php:99
    0.4046   11902112   4. MediaWikiPHPUnitCommand->run(???, ???) /var/www/mediawiki/tests/phpunit/phpunit.php:90
    0.4508   12630272   5. MediaWikiTestRunner->getTest(???, ???, ???) /var/www/mediawiki/vendor/phpunit/phpunit/src/TextUI/Command.php:169
    0.4511   12630272   6. MediaWikiTestRunner->loadSuiteClass(???, ???) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:73
    0.4527   12647448   7. PHPUnit\Runner\StandardTestSuiteLoader->load(???, ???) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:130
    0.4529   12667984   8. PHPUnit\Util\Fileloader::checkAndLoad(???) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php:43
    0.4534   12668240   9. PHPUnit\Util\Fileloader::load(???) /var/www/mediawiki/vendor/phpunit/phpunit/src/Util/Fileloader.php:48
    0.4541   12679072  10. include_once('/var/www/mediawiki/tests/phpunit/includes/search/SearchResultTest.php') /var/www/mediawiki/vendor/phpunit/phpunit/src/Util/Fileloader.php:64

Using PHP 7.2.14
PHP Fatal error:  Class 'MediawikiTestCase' not found in /var/www/mediawiki/tests/phpunit/includes/search/SearchResultTest.php on line 3
PHP Stack trace:
PHP   1. {main}() /var/www/mediawiki/tests/phpunit/phpunit.php:0
PHP   2. require() /var/www/mediawiki/tests/phpunit/phpunit.php:129
PHP   3. PHPUnitMaintClass->execute() /var/www/mediawiki/maintenance/doMaintenance.php:99
PHP   4. MediaWikiPHPUnitCommand->run($argv = *uninitialized*, $exit = *uninitialized*) /var/www/mediawiki/tests/phpunit/phpunit.php:90
PHP   5. MediaWikiTestRunner->getTest($suiteClassName = *uninitialized*, $suiteClassFile = *uninitialized*, $suffixes = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/TextUI/Command.php:169
PHP   6. MediaWikiTestRunner->loadSuiteClass($suiteClassName = *uninitialized*, $suiteClassFile = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:73
PHP   7. PHPUnit\Runner\StandardTestSuiteLoader->load($suiteClassName = *uninitialized*, $suiteClassFile = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:130
PHP   8. PHPUnit\Util\Fileloader::checkAndLoad($filename = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php:43
PHP   9. PHPUnit\Util\Fileloader::load($filename = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/Util/Fileloader.php:48
PHP  10. include_once() /var/www/mediawiki/vendor/phpunit/phpunit/src/Util/Fileloader.php:64

[1] tests/phpunit/includes/libs/GenericArrayObjectTest.phpPHPUnit\Runner\Exception from line 102 of /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php: Class 'tests/phpunit/includes/libs/GenericArrayObjectTest' could not be found in '/var/www/mediawiki/tests/phpunit/includes/libs/GenericArrayObjectTest.php'.
#0 /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php(130): PHPUnit\Runner\StandardTestSuiteLoader->load('tests/phpunit/i...', '/var/www/mediaw...')
#1 /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php(73): PHPUnit\Runner\BaseTestRunner->loadSuiteClass('tests/phpunit/i...', '/var/www/mediaw...')
#2 /var/www/mediawiki/vendor/phpunit/phpunit/src/TextUI/Command.php(169): PHPUnit\Runner\BaseTestRunner->getTest('tests/phpunit/i...', '/var/www/mediaw...', Array)
#3 /var/www/mediawiki/tests/phpunit/phpunit.php(90): PHPUnit\TextUI\Command->run(Array, true)
#4 /var/www/mediawiki/maintenance/doMaintenance.php(99): PHPUnitMaintClass->execute()
#5 /var/www/mediawiki/tests/phpunit/phpunit.php(129): require('/var/www/mediaw...')
#6 {main}
Using PHP 7.2.14

[3] tests/phpunit/suites/ParserIntegrationTest.phpUsing PHP 7.2.14
[4ca2f19e580cc04e08f5ae71] [no req]   ArgumentCountError from line 44 of /var/www/mediawiki/tests/phpunit/suites/ParserIntegrationTest.php: Too few arguments to function ParserIntegrationTest::__construct(), 0 passed in /var/www/mediawiki/vendor/phpunit/phpunit/src/Framework/TestSuite.php on line 591 and exactly 3 expected
Backtrace:
#0 /var/www/mediawiki/vendor/phpunit/phpunit/src/Framework/TestSuite.php(591): ParserIntegrationTest->__construct()
#1 /var/www/mediawiki/vendor/phpunit/phpunit/src/Framework/TestSuite.php(882): PHPUnit\Framework\TestSuite::createTest(ReflectionClass, string)
#2 /var/www/mediawiki/vendor/phpunit/phpunit/src/Framework/TestSuite.php(187): PHPUnit\Framework\TestSuite->addTestMethod(ReflectionClass, ReflectionMethod)
#3 /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php(106): PHPUnit\Framework\TestSuite->__construct(ReflectionClass)
#4 /var/www/mediawiki/vendor/phpunit/phpunit/src/TextUI/Command.php(169): PHPUnit\Runner\BaseTestRunner->getTest(string, string, array)
#5 /var/www/mediawiki/tests/phpunit/phpunit.php(90): PHPUnit\TextUI\Command->run(array, boolean)
#6 /var/www/mediawiki/maintenance/doMaintenance.php(99): PHPUnitMaintClass->execute()
#7 /var/www/mediawiki/tests/phpunit/phpunit.php(129): require(string)
#8 {main}

[3] tests/phpunit/LessFileCompilationTest.phpUsing PHP 7.2.14
[b865182a94074440ab541361] [no req]   ArgumentCountError from line 28 of /var/www/mediawiki/tests/phpunit/LessFileCompilationTest.php: Too few arguments to function LessFileCompilationTest::__construct(), 0 passed in /var/www/mediawiki/vendor/phpunit/phpunit/src/Framework/TestSuite.php on line 591 and exactly 2 expected
Backtrace:
#0 /var/www/mediawiki/vendor/phpunit/phpunit/src/Framework/TestSuite.php(591): LessFileCompilationTest->__construct()
#1 /var/www/mediawiki/vendor/phpunit/phpunit/src/Framework/TestSuite.php(882): PHPUnit\Framework\TestSuite::createTest(ReflectionClass, string)
#2 /var/www/mediawiki/vendor/phpunit/phpunit/src/Framework/TestSuite.php(187): PHPUnit\Framework\TestSuite->addTestMethod(ReflectionClass, ReflectionMethod)
#3 /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php(106): PHPUnit\Framework\TestSuite->__construct(ReflectionClass)
#4 /var/www/mediawiki/vendor/phpunit/phpunit/src/TextUI/Command.php(169): PHPUnit\Runner\BaseTestRunner->getTest(string, string, array)
#5 /var/www/mediawiki/tests/phpunit/phpunit.php(90): PHPUnit\TextUI\Command->run(array, boolean)
#6 /var/www/mediawiki/maintenance/doMaintenance.php(99): PHPUnitMaintClass->execute()
#7 /var/www/mediawiki/tests/phpunit/phpunit.php(129): require(string)
#8 {main}

[4] tests/phpunit/docs/ExportDemoTest.php
Fatal error: Class 'DumpTestCase' not found in /var/www/mediawiki/tests/phpunit/docs/ExportDemoTest.php on line 11

Call Stack:
    0.0118     406360   1. {main}() /var/www/mediawiki/tests/phpunit/phpunit.php:0
    0.0490     698768   2. require('/var/www/mediawiki/maintenance/doMaintenance.php') /var/www/mediawiki/tests/phpunit/phpunit.php:129
    1.0348   11037104   3. PHPUnitMaintClass->execute() /var/www/mediawiki/maintenance/doMaintenance.php:99
    1.0749   11902080   4. MediaWikiPHPUnitCommand->run(???, ???) /var/www/mediawiki/tests/phpunit/phpunit.php:90
    1.2131   12630224   5. MediaWikiTestRunner->getTest(???, ???, ???) /var/www/mediawiki/vendor/phpunit/phpunit/src/TextUI/Command.php:169
    1.2143   12630224   6. MediaWikiTestRunner->loadSuiteClass(???, ???) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:73
    1.2188   12647400   7. PHPUnit\Runner\StandardTestSuiteLoader->load(???, ???) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:130
    1.2191   12667936   8. PHPUnit\Util\Fileloader::checkAndLoad(???) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php:43
    1.2214   12668192   9. PHPUnit\Util\Fileloader::load(???) /var/www/mediawiki/vendor/phpunit/phpunit/src/Util/Fileloader.php:48
    1.2236   12672704  10. include_once('/var/www/mediawiki/tests/phpunit/docs/ExportDemoTest.php') /var/www/mediawiki/vendor/phpunit/phpunit/src/Util/Fileloader.php:64

Using PHP 7.2.14
PHP Fatal error:  Class 'DumpTestCase' not found in /var/www/mediawiki/tests/phpunit/docs/ExportDemoTest.php on line 11
PHP Stack trace:
PHP   1. {main}() /var/www/mediawiki/tests/phpunit/phpunit.php:0
PHP   2. require() /var/www/mediawiki/tests/phpunit/phpunit.php:129
PHP   3. PHPUnitMaintClass->execute() /var/www/mediawiki/maintenance/doMaintenance.php:99
PHP   4. MediaWikiPHPUnitCommand->run($argv = *uninitialized*, $exit = *uninitialized*) /var/www/mediawiki/tests/phpunit/phpunit.php:90
PHP   5. MediaWikiTestRunner->getTest($suiteClassName = *uninitialized*, $suiteClassFile = *uninitialized*, $suffixes = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/TextUI/Command.php:169
PHP   6. MediaWikiTestRunner->loadSuiteClass($suiteClassName = *uninitialized*, $suiteClassFile = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:73
PHP   7. PHPUnit\Runner\StandardTestSuiteLoader->load($suiteClassName = *uninitialized*, $suiteClassFile = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:130
PHP   8. PHPUnit\Util\Fileloader::checkAndLoad($filename = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/Runner/StandardTestSuiteLoader.php:43
PHP   9. PHPUnit\Util\Fileloader::load($filename = *uninitialized*) /var/www/mediawiki/vendor/phpunit/phpunit/src/Util/Fileloader.php:48
PHP  10. include_once() /var/www/mediawiki/vendor/phpunit/phpunit/src/Util/Fileloader.php:64

[4] tests/phpunit/includes/search/SearchEngineTest.phpUsing PHP 7.2.14
PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

.........F

Time: 38.75 seconds, Memory: 34.00MB

There was 1 failure:

1) SearchEngineTest::testCompletionSearchMustRespectCapitalLinkOverrides with data set "Searching for "search is" will finds "search is not Search" on NS_CATEGORY" ('search is', 'Category:search is not Search', array(14))
=== Logs generated by test case
[caches] [info] LocalisationCache: using store LCStoreNull {"private":false}
[wfDebug] [debug] LocalisationCache::isExpired(en): cache missing, need to make one {"private":false}
[wfDebug] [debug] LocalisationCache::recache: got localisation for en from source {"private":false}
[caches] [info] LocalisationCache: using store LCStoreNull {"private":false}
===
Failed asserting that 0 matches expected 1.

/var/www/mediawiki/tests/phpunit/includes/search/SearchEngineTest.php:265
/var/www/mediawiki/tests/phpunit/MediaWikiTestCase.php:427
/var/www/mediawiki/maintenance/doMaintenance.php:99

FAILURES!
Tests: 10, Assertions: 34, Failures: 1.


    ✘ ehm broken tests...
    Time: 39 minutes 21 seconds 130 milliseconds, Memory: 6.00 MiB

In our CI environment where we have 8 vCPUs, this might go a lot faster.

There are some failures above due to expectation of running tests serially (e.g. testMediaWikiTestCaseSchemaTestOrder) but overall the test failures here don't seem to be insurmountable.

This might make more sense to pursue further after T90875: Use vendor/bin/phpunit instead of tests/phpunit/phpunit.php is done.

My rough plan would be:

  1. Patch quibble so that it accepts a --phpunit-command argument. (That will also be needed for T90875, so that you could run quibble --phpunt-command=vendor/bin/phpunit.)
  2. Submit a patch for integration/config with quibble --phpunit-command='find tests/phpunit extensions/**/tests/phpunit skins/**/tests/phpunit -name "*Test.php" | ./vendor/liuggio/fastest/fastest "tests/phpunit/phpunit.php {};"'
  3. Create a patch for core which adds https://github.com/liuggio/fastest to require-dev
  4. Pair with someone to do a test run in CI to check speed
kostajh renamed this task from Speed up MediaWiki PHPUnit build by running tests in parallel to Speed up MediaWiki PHPUnit build by running integration tests in parallel.Jun 17 2019, 5:55 PM
kostajh added subscribers: Ladsgroup, awight.
Krinkle mentioned this in T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min.Jun 17 2019, 5:57 PM
hashar added a comment.Jun 19 2019, 7:05 AM

There are a few gotchas though. Some testsuite define tests which would not be found by globbing files. Typically the parser tests. There are also alternative ways to register test files via the UnitTest hook.

running the full test suite takes 49 minutes. With fastest, it takes 39 minutes:

That suggest the suite is not that much CPU bound or surely running four in parallels would be way faster?

root@68e6b761fef5:/var/www/mediawiki# find tests/phpunit extensions/**/tests/phpunit skins/**/tests/phpunit -name "*Test.php" | ./vendor/liuggio/fastest/fastest "tests/phpunit/phpunit.php {};"
find: ‘extensions/**/tests/phpunit’: No such file or directory
find: ‘skins/**/tests/phpunit’: No such file or directory

From the command line, that seems to suggest to run test classes from mediawiki/core as well as all extensions and skins. Then it does not find any file for extensions or skins. So is that solely for mediawiki/core tests? And if so why does it take 49 minutes when being run serially? It should be ten time faster (3 mins 15 s on my computer with hhvm).

In our CI environment where we have 8 vCPUs, this might go a lot faster.

The devil there is that we do not have the capacity to add 8 times more CPU on the WMCS infra. Even if instances have 8 vCPUS, they currently can run up to 4 jobs in parallel and we still need lot of CPU cycles for eg MySQL :]

Krinkle moved this task from Inbox to PHPUnit on the MediaWiki-Core-Tests board.Sep 4 2019, 5:49 PM
kostajh mentioned this in T235449: Quibble: Run PHPUnit databaseless and database stages in parallel.Oct 14 2019, 7:39 PM
Daimona added a subscriber: Daimona.Jan 21 2020, 3:23 PM
gerritbot added a comment.Mar 19 2020, 12:48 AM

Change 581131 had a related patch set uploaded (by Krinkle; owner: Aaron Schulz):
[mediawiki/core@master] tests: Split includes/ tests into more suites for tools like paratest

https://gerrit.wikimedia.org/r/581131

gerritbot added a project: Patch-For-Review.Mar 19 2020, 12:48 AM
gerritbot added a comment.Oct 29 2020, 10:20 PM

Change 637574 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] Move ApiQuery* tests under the /query subdirectory

https://gerrit.wikimedia.org/r/637574

gerritbot added a comment.Nov 4 2020, 6:07 AM

Change 637574 merged by jenkins-bot:
[mediawiki/core@master] Move ApiQuery* tests under the /query subdirectory

https://gerrit.wikimedia.org/r/637574

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.18; 2020-11-17).Nov 4 2020, 7:00 AM
Krinkle mentioned this in T90875: Use vendor/bin/phpunit instead of tests/phpunit/phpunit.php.Mar 15 2021, 5:43 PM
kostajh updated the task description. (Show Details)Mar 15 2021, 8:41 PM
hashar removed projects: MW-1.36-notes (1.36.0-wmf.18; 2020-11-17), Patch-For-Review, Continuous-Integration-Config.May 21 2021, 2:09 PM
Krinkle merged a task: T269894: Include paratest runner with stock MediaWiki.Aug 31 2021, 1:22 AM
Krinkle added subscribers: aaron, thcipriani.
@aaron wrote at T269894

Something like the phpunit/paratest wrapper at https://github.com/AaronSchulz/mediawiki-developer-tools would be nice to have in MediaWiki core.

  • Merge mediawiki-developer-tools suite.xml optimizations (or an improved version) into core (these breaks up the tests into more suites that can be run in parallel)
  • Add support for cloning to new sqlite DB files in CloneDatabase; this will replace wfCloneSqliteSchemaForParatest() from mediawiki-developer-tools
  • Add paratest to composer.json
  • Merge ParatestWrapperSettings.php into core from mediawiki-developer-tools
  • Include a simplified version of mediawiki-phpunit from mediawiki-developer-tools in core (e.g. just for "dev_phpunit" and "dev_paratest" modes)
Krinkle added a parent task: T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min.Aug 31 2021, 1:22 AM
gerritbot added a comment.Nov 27 2021, 2:02 PM

Change 742199 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] phpunit: Support splitting extension suite

https://gerrit.wikimedia.org/r/742199

gerritbot added a project: Patch-For-Review.Nov 27 2021, 2:02 PM
gerritbot added a comment.Nov 27 2021, 2:05 PM

Change 742200 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/quibble@master] [WIP] Run PHPUnit tests in parallel

https://gerrit.wikimedia.org/r/742200

Krinkle added a subtask: T297078: PHPUnit tests should pass when run on their own.Dec 6 2021, 6:07 PM
kostajh added a comment.Dec 6 2021, 8:22 PM
In T50217#7531811, @gerritbot wrote:

Change 742199 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] phpunit: Support splitting extension suite

https://gerrit.wikimedia.org/r/742199

In T50217#7531813, @gerritbot wrote:

Change 742200 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/quibble@master] [WIP] Run PHPUnit tests in parallel

https://gerrit.wikimedia.org/r/742200

To explain a bit about what I'm trying here:

  • In the short term, I'm skeptical about adding paratest or other wrappers like https://github.com/liuggio/fastest until T90875: Use vendor/bin/phpunit instead of tests/phpunit/phpunit.php is done; it seems like adding more code to wrap around tests/phpunit/phpunit.php will make removing it in favor of vendor/bin/phpunit that much harder. Longer term, having paratest would be nice to faciliate local test execution.
  • we have some parallelism utility code in Quibble that seemed simple to adapt for PHPUnit. In fact, it is quite easy to adapt, but it turns out that a bunch of our tests don't pass when run out of order, on their own, or even when they are executed with --testsuite={testSuiteName} ==testsuite={someOtherTestSuiteName}. So as @Krinkle linked to above, I'm trying to fix tests and making subtasks as needed under T297078: PHPUnit tests should pass when run on their own.
kostajh added a comment.Dec 6 2021, 8:43 PM
In T50217#7551037, @kostajh wrote:
In T50217#7531811, @gerritbot wrote:

Change 742199 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] phpunit: Support splitting extension suite

https://gerrit.wikimedia.org/r/742199

In T50217#7531813, @gerritbot wrote:

Change 742200 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/quibble@master] [WIP] Run PHPUnit tests in parallel

https://gerrit.wikimedia.org/r/742200

To explain a bit about what I'm trying here:

  • In the short term, I'm skeptical about adding paratest or other wrappers like https://github.com/liuggio/fastest until T90875: Use vendor/bin/phpunit instead of tests/phpunit/phpunit.php is done; it seems like adding more code to wrap around tests/phpunit/phpunit.php will make removing it in favor of vendor/bin/phpunit that much harder. Longer term, having paratest would be nice to faciliate local test execution.
  • we have some parallelism utility code in Quibble that seemed simple to adapt for PHPUnit. In fact, it is quite easy to adapt, but it turns out that a bunch of our tests don't pass when run out of order, on their own, or even when they are executed with --testsuite={testSuiteName} ==testsuite={someOtherTestSuiteName}. So as @Krinkle linked to above, I'm trying to fix tests and making subtasks as needed under T297078: PHPUnit tests should pass when run on their own.

The last ones to fix (which are also harder because I can't reproduce them breaking when running locally) from https://integration.wikimedia.org/ci/job/integration-quibble-fullrun/504/console:

21:31:19 Time: 1.64 minutes, Memory: 904.50 MB
21:31:19 
21:31:19 There was 1 error:
21:31:19 
21:31:19 1) MediaWikiServicesTest::testDefaultServiceInstantiation
21:31:19 Use of GrowthExperimentsTaskSuggester service was deprecated in GrowthExperiments 1.35. [Called from Wikimedia\Services\ServiceContainer::createService in /workspace/src/vendor/wikimedia/services/src/ServiceContainer.php at line 447]
21:31:19 
21:31:19 /workspace/src/includes/debug/MWDebug.php:377
21:31:19 /workspace/src/includes/debug/MWDebug.php:353
21:31:19 /workspace/src/includes/debug/MWDebug.php:232
21:31:19 /workspace/src/includes/GlobalFunctions.php:1005
21:31:19 /workspace/src/extensions/GrowthExperiments/ServiceWiring.php:589
21:31:19 /workspace/src/vendor/wikimedia/services/src/ServiceContainer.php:447
21:31:19 /workspace/src/vendor/wikimedia/services/src/ServiceContainer.php:416
21:31:19 /workspace/src/includes/MediaWikiServices.php:291
21:31:19 /workspace/src/tests/phpunit/includes/MediaWikiServicesTest.php:386
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 [localisation] [debug] LocalisationCache using store LCStoreNull []
21:31:19 [objectcache] [debug] MainObjectStash using store {class} {"class":"HashBagOStuff"}
21:31:19 [GlobalTitleFail] [info] RequestContext::getTitle called with no title set. {"exception":{}}
21:31:19 [localisation] [debug] LocalisationCache::isExpired(en): cache missing, need to make one []
21:31:19 [GlobalTitleFail] [info] RequestContext::getTitle called with no title set. {"exception":{}}
21:31:19 [GlobalTitleFail] [info] RequestContext::getTitle called with no title set. {"exception":{}}
21:31:19 [wfDebug] [debug] ParserFactory: using default preprocessor {"private":false}
21:31:19 [ParserCache] [debug] Creating ParserCache instance for pcache []
21:31:19 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"}
21:31:19 [objectcache] [debug] fetchOrRegenerate(wikidbA-unittest_:page:8:0e6409a4846e9aee0a73fd5dfb53dd5e9839c4c6): miss, new value computed []
21:31:19 ===
21:31:19 
21:31:19 --
21:31:19 
21:31:19 There were 12 failures:
21:31:19 
21:31:19 1) MediaWiki\Tests\Revision\RevisionRendererTest::testGetRenderedRevision_multi
21:31:19 slot header
21:31:19 Failed asserting that '<div class="mw-parser-output"><p><a href="/index.php?title=Kittens&amp;action=edit&amp;redlink=1" class="new" title="Kittens (page does not exist)">Kittens</a>\n
21:31:19 </p><h1 class="mw-slot-header"><mediainfoslotheader /></h1><p><a href="/index.php?title=Goats&amp;action=edit&amp;redlink=1" class="new" title="Goats (page does not exist)">Goats</a>\n
21:31:19 </p></div>' contains ">aux<".
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/Revision/RevisionRendererTest.php:420
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 [localisation] [debug] LocalisationCache using store LCStoreNull []
21:31:19 [objectcache] [debug] MainObjectStash using store {class} {"class":"HashBagOStuff"}
21:31:19 [ContentHandler] [info] Registered handler for wikitext: WikitextContentHandler []
21:31:19 [localisation] [debug] LocalisationCache::isExpired(en): cache missing, need to make one []
21:31:19 [wfDebug] [debug] ParserFactory: using default preprocessor {"private":false}
21:31:19 [Wikibase] [debug] {method}: setting {settingName} was given as a closure, resolve it to {logValue} {"method":"Wikibase\\Lib\\SettingsArray::getSetting","settingName":"thisWikiIsTheRepo","logValue":"true"}
21:31:19 [Wikibase] [debug] {method}: setting {settingName} was given as a closure, resolve it to {logValue} {"method":"Wikibase\\Lib\\SettingsArray::getSetting","settingName":"entitySources","logValue":"array (\n  'local' => \n  array (\n    'entityNamespaces' => \n    array (\n      'item' => 120,\n      'property' => 122,\n      'mediainfo' => '6\/mediainfo',\n    ),\n    'repoDatabase' => false,\n    'baseUri' => 'http:\/\/127.0.0.1:9412\/entity\/',\n    'rdfNodeNamespacePrefix' => 'wd',\n    'rdfPredicateNamespacePrefix' => '',\n    'interwikiPrefix' => '',\n  ),\n)"}
21:31:19 [Wikibase] [debug] {method}: setting {settingName} was given as a closure, resolve it to {logValue} {"method":"Wikibase\\Lib\\SettingsArray::getSetting","settingName":"entitySources","logValue":"array (\n  'local' => \n  array (\n    'repoDatabase' => false,\n    'entityNamespaces' => \n    array (\n      'item' => '120\/main',\n      'property' => '122\/main',\n      'mediainfo' => '6\/mediainfo',\n    ),\n    'baseUri' => 'http:\/\/127.0.0.1:9412\/entity\/',\n    'rdfNodeNamespacePrefix' => 'wd',\n    'rdfPredicateNamespacePrefix' => '',\n    'interwikiPrefix' => '',\n  ),\n)"}
21:31:19 [Wikibase] [debug] {method}: setting {settingName} was given as a closure, resolve it to {logValue} {"method":"Wikibase\\Lib\\SettingsArray::getSetting","settingName":"itemAndPropertySourceName","logValue":"'local'"}
21:31:19 [Wikibase] [debug] {method}: setting {settingName} was given as a closure, resolve it to {logValue} {"method":"Wikibase\\Lib\\SettingsArray::getSetting","settingName":"siteGlobalID","logValue":"'wikidbA'"}
21:31:19 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"}
21:31:19 [Wikibase] [debug] {method}: setting {settingName} was given as a closure, resolve it to {logValue} {"method":"Wikibase\\Lib\\SettingsArray::getSetting","settingName":"siteGroup","logValue":"NULL"}
21:31:19 [DuplicateParse] [debug] MediaWiki\Parser\ParserObserver::notifyParse: Possibly redundant parse! {"page":"ns0:MediaWiki\\Tests\\Revision\\RevisionRendererTest","rev":null,"options-hash":"canonical","trace":"#0 \/workspace\/src\/includes\/content\/ContentHandler.php(1728): MediaWiki\\Parser\\ParserObserver->notifyParse(Object(Title), NULL, Object(ParserOptions), Object(ParserOutput))\n#1 \/workspace\/src\/includes\/content\/Renderer\/ContentRenderer.php(47): ContentHandler->getParserOutput(Object(WikitextContent), Object(MediaWiki\\Content\\Renderer\\ContentParseParams))\n#2 \/workspace\/src\/includes\/Revision\/RenderedRevision.php(271): MediaWiki\\Content\\Renderer\\ContentRenderer->getParserOutput(Object(WikitextContent), Object(MediaWiki\\Page\\PageIdentityValue), NULL, Object(ParserOptions), true)\n#3 \/workspace\/src\/includes\/Revision\/RenderedRevision.php(238): MediaWiki\\Revision\\RenderedRevision->getSlotParserOutputUncached(Object(WikitextContent), true)\n#4 \/workspace\/src\/includes\/Revision\/RevisionRenderer.php(236): MediaWiki\\Revision\\RenderedRevision->getSlotParserOutput('aux', Array)\n#5 \/workspace\/src\/includes\/Revision\/RevisionRenderer.php(158): MediaWiki\\Revision\\RevisionRenderer->combineSlotOutput(Object(MediaWiki\\Revision\\RenderedRevision), Array)\n#6 [internal function]: MediaWiki\\Revision\\RevisionRenderer->MediaWiki\\Revision\\{closure}(Object(MediaWiki\\Revision\\RenderedRevision), Array)\n#7 \/workspace\/src\/includes\/Revision\/RenderedRevision.php(200): call_user_func(Object(Closure), Object(MediaWiki\\Revision\\RenderedRevision), Array)\n#8 \/workspace\/src\/tests\/phpunit\/includes\/Revision\/RevisionRendererTest.php(406): MediaWiki\\Revision\\RenderedRevision->getRevisionParserOutput()\n#9 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestCase.php(1472): MediaWiki\\Tests\\Revision\\RevisionRendererTest->testGetRenderedRevision_multi()\n#10 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestCase.php(1092): PHPUnit\\Framework\\TestCase->runTest()\n#11 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestResult.php(703): PHPUnit\\Framework\\TestCase->runBare()\n#12 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestCase.php(820): PHPUnit\\Framework\\TestResult->run(Object(MediaWiki\\Tests\\Revision\\RevisionRendererTest))\n#13 \/workspace\/src\/tests\/phpunit\/MediaWikiIntegrationTestCase.php(456): PHPUnit\\Framework\\TestCase->run(Object(PHPUnit\\Framework\\TestResult))\n#14 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestSuite.php(627): MediaWikiIntegrationTestCase->run(Object(PHPUnit\\Framework\\TestResult))\n#15 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestSuite.php(627): PHPUnit\\Framework\\TestSuite->run(Object(PHPUnit\\Framework\\TestResult))\n#16 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestSuite.php(627): PHPUnit\\Framework\\TestSuite->run(Object(PHPUnit\\Framework\\TestResult))\n#17 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/TextUI\/TestRunner.php(656): PHPUnit\\Framework\\TestSuite->run(Object(PHPUnit\\Framework\\TestResult))\n#18 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/TextUI\/Command.php(235): PHPUnit\\TextUI\\TestRunner->doRun(Object(PHPUnit\\Framework\\TestSuite), Array, Array, true)\n#19 \/workspace\/src\/tests\/phpunit\/phpunit.php(141): PHPUnit\\TextUI\\Command->run(Array, true)\n#20 \/workspace\/src\/tests\/phpunit\/phpunit.php(207): PHPUnitMaintClass->execute()\n#21 {main}","previous-trace":"#0 \/workspace\/src\/includes\/content\/ContentHandler.php(1728): MediaWiki\\Parser\\ParserObserver->notifyParse(Object(Title), NULL, Object(ParserOptions), Object(ParserOutput))\n#1 \/workspace\/src\/includes\/content\/Renderer\/ContentRenderer.php(47): ContentHandler->getParserOutput(Object(WikitextContent), Object(MediaWiki\\Content\\Renderer\\ContentParseParams))\n#2 \/workspace\/src\/includes\/Revision\/RenderedRevision.php(271): MediaWiki\\Content\\Renderer\\ContentRenderer->getParserOutput(Object(WikitextContent), Object(MediaWiki\\Page\\PageIdentityValue), NULL, Object(ParserOptions), true)\n#3 \/workspace\/src\/includes\/Revision\/RenderedRevision.php(238): MediaWiki\\Revision\\RenderedRevision->getSlotParserOutputUncached(Object(WikitextContent), true)\n#4 \/workspace\/src\/includes\/Revision\/RevisionRenderer.php(236): MediaWiki\\Revision\\RenderedRevision->getSlotParserOutput('main', Array)\n#5 \/workspace\/src\/includes\/Revision\/RevisionRenderer.php(158): MediaWiki\\Revision\\RevisionRenderer->combineSlotOutput(Object(MediaWiki\\Revision\\RenderedRevision), Array)\n#6 [internal function]: MediaWiki\\Revision\\RevisionRenderer->MediaWiki\\Revision\\{closure}(Object(MediaWiki\\Revision\\RenderedRevision), Array)\n#7 \/workspace\/src\/includes\/Revision\/RenderedRevision.php(200): call_user_func(Object(Closure), Object(MediaWiki\\Revision\\RenderedRevision), Array)\n#8 \/workspace\/src\/tests\/phpunit\/includes\/Revision\/RevisionRendererTest.php(406): MediaWiki\\Revision\\RenderedRevision->getRevisionParserOutput()\n#9 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestCase.php(1472): MediaWiki\\Tests\\Revision\\RevisionRendererTest->testGetRenderedRevision_multi()\n#10 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestCase.php(1092): PHPUnit\\Framework\\TestCase->runTest()\n#11 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestResult.php(703): PHPUnit\\Framework\\TestCase->runBare()\n#12 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestCase.php(820): PHPUnit\\Framework\\TestResult->run(Object(MediaWiki\\Tests\\Revision\\RevisionRendererTest))\n#13 \/workspace\/src\/tests\/phpunit\/MediaWikiIntegrationTestCase.php(456): PHPUnit\\Framework\\TestCase->run(Object(PHPUnit\\Framework\\TestResult))\n#14 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestSuite.php(627): MediaWikiIntegrationTestCase->run(Object(PHPUnit\\Framework\\TestResult))\n#15 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestSuite.php(627): PHPUnit\\Framework\\TestSuite->run(Object(PHPUnit\\Framework\\TestResult))\n#16 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/Framework\/TestSuite.php(627): PHPUnit\\Framework\\TestSuite->run(Object(PHPUnit\\Framework\\TestResult))\n#17 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/TextUI\/TestRunner.php(656): PHPUnit\\Framework\\TestSuite->run(Object(PHPUnit\\Framework\\TestResult))\n#18 \/workspace\/src\/vendor\/phpunit\/phpunit\/src\/TextUI\/Command.php(235): PHPUnit\\TextUI\\TestRunner->doRun(Object(PHPUnit\\Framework\\TestSuite), Array, Array, true)\n#19 \/workspace\/src\/tests\/phpunit\/phpunit.php(141): PHPUnit\\TextUI\\Command->run(Array, true)\n#20 \/workspace\/src\/tests\/phpunit\/phpunit.php(207): PHPUnitMaintClass->execute()\n#21 {main}"}
21:31:19 ===
21:31:19 
21:31:19 2) MediaWiki\Tests\Parser\ParserCacheTest::testJsonEncodeUnicode
21:31:19 Failed asserting that '' contains "Э".
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/parser/ParserCacheTest.php:762
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 [localisation] [debug] LocalisationCache using store LCStoreNull []
21:31:19 [objectcache] [debug] MainObjectStash using store {class} {"class":"HashBagOStuff"}
21:31:19 [ContentHandler] [info] Registered handler for wikitext: WikitextContentHandler []
21:31:19 ===
21:31:19 
21:31:19 3) NamespaceInfoTest::testGetCanonicalNamespaces
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19      3 => 'User_talk'
21:31:19      -1 => 'Special'
21:31:19      -2 => 'Media'
21:31:19 +    250 => 'Page'
21:31:19 +    251 => 'Page_talk'
21:31:19 +    252 => 'Index'
21:31:19 +    253 => 'Index_talk'
21:31:19 +    710 => 'TimedText'
21:31:19 +    711 => 'TimedText_talk'
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:842
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 4) NamespaceInfoTest::testGetValidNamespaces
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19      1 => 1
21:31:19      2 => 2
21:31:19      3 => 3
21:31:19 +    4 => 250
21:31:19 +    5 => 251
21:31:19 +    6 => 252
21:31:19 +    7 => 253
21:31:19 +    8 => 710
21:31:19 +    9 => 711
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:896
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 5) NamespaceInfoTest::testGetCanonicalNamespaces_NoCanonicalNamespaceNames
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19  Array &0 (
21:31:19      0 => ''
21:31:19 +    250 => 'Page'
21:31:19 +    251 => 'Page_talk'
21:31:19 +    252 => 'Index'
21:31:19 +    253 => 'Index_talk'
21:31:19 +    710 => 'TimedText'
21:31:19 +    711 => 'TimedText_talk'
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:911
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 6) NamespaceInfoTest::testGetValidNamespaces_NoCanonicalNamespaceNames
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19  Array &0 (
21:31:19      0 => 0
21:31:19 +    1 => 250
21:31:19 +    2 => 251
21:31:19 +    3 => 252
21:31:19 +    4 => 253
21:31:19 +    5 => 710
21:31:19 +    6 => 711
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:940
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 7) NamespaceInfoTest::testGetCanonicalNamespaces_ExtensionNamespaces
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19      -1 => 'Special'
21:31:19      -2 => 'Media'
21:31:19      12345 => 'Extended'
21:31:19 +    250 => 'Page'
21:31:19 +    251 => 'Page_talk'
21:31:19 +    252 => 'Index'
21:31:19 +    253 => 'Index_talk'
21:31:19 +    710 => 'TimedText'
21:31:19 +    711 => 'TimedText_talk'
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:963
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 8) NamespaceInfoTest::testGetValidNamespaces_ExtensionNamespaces
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19      1 => 1
21:31:19      2 => 2
21:31:19      3 => 3
21:31:19 -    4 => 12345
21:31:19 +    4 => 250
21:31:19 +    5 => 251
21:31:19 +    6 => 252
21:31:19 +    7 => 253
21:31:19 +    8 => 710
21:31:19 +    9 => 711
21:31:19 +    10 => 12345
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:999
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 9) NamespaceInfoTest::testGetCanonicalNamespaces_ExtraNamespaces
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19      -1 => 'Special'
21:31:19      -2 => 'Media'
21:31:19      1234567 => 'Extra'
21:31:19 +    250 => 'Page'
21:31:19 +    251 => 'Page_talk'
21:31:19 +    252 => 'Index'
21:31:19 +    253 => 'Index_talk'
21:31:19 +    710 => 'TimedText'
21:31:19 +    711 => 'TimedText_talk'
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:1089
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 10) NamespaceInfoTest::testGetValidNamespaces_ExtraNamespaces
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19      1 => 1
21:31:19      2 => 2
21:31:19      3 => 3
21:31:19 -    4 => 1234567
21:31:19 +    4 => 250
21:31:19 +    5 => 251
21:31:19 +    6 => 252
21:31:19 +    7 => 253
21:31:19 +    8 => 710
21:31:19 +    9 => 711
21:31:19 +    10 => 1234567
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:1121
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 11) NamespaceInfoTest::testGetCanonicalNamespaces_caching
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19      3 => 'User_talk'
21:31:19      -1 => 'Special'
21:31:19      -2 => 'Media'
21:31:19 +    250 => 'Page'
21:31:19 +    251 => 'Page_talk'
21:31:19 +    252 => 'Index'
21:31:19 +    253 => 'Index_talk'
21:31:19 +    710 => 'TimedText'
21:31:19 +    711 => 'TimedText_talk'
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:1144
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
21:31:19 
21:31:19 12) NamespaceInfoTest::testGetValidNamespaces_caching
21:31:19 Failed asserting that two arrays are identical.
21:31:19 --- Expected
21:31:19 +++ Actual
21:31:19 @@ @@
21:31:19      1 => 1
21:31:19      2 => 2
21:31:19      3 => 3
21:31:19 +    4 => 250
21:31:19 +    5 => 251
21:31:19 +    6 => 252
21:31:19 +    7 => 253
21:31:19 +    8 => 710
21:31:19 +    9 => 711
21:31:19  )
21:31:19 
21:31:19 /workspace/src/tests/phpunit/includes/title/NamespaceInfoTest.php:1203
21:31:19 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:456
21:31:19 /workspace/src/tests/phpunit/phpunit.php:141
21:31:19 /workspace/src/tests/phpunit/phpunit.php:207
21:31:19 === Logs generated by test case
21:31:19 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
21:31:19 ===
gerritbot added a comment.Feb 1 2022, 8:26 AM

Change 758784 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] phpunit: Use process ID in test DB prefix

https://gerrit.wikimedia.org/r/758784

gerritbot added a comment.Feb 22 2022, 8:52 PM

Change 581131 abandoned by Aaron Schulz:

[mediawiki/core@master] tests: split includes/ tests into suites for tools like paratest

Reason:

Going with automatic generation from paratest wrapper

https://gerrit.wikimedia.org/r/581131

gerritbot added a comment.Feb 24 2022, 4:51 PM

Change 763650 had a related patch set uploaded (by Krinkle; author: Aaron Schulz):

[mediawiki/core@master] phpunit: Remove redundancy from \"skins\" test and \"extensions\" suites

https://gerrit.wikimedia.org/r/763650

gerritbot added a comment.Mar 1 2022, 9:57 PM

Change 763650 abandoned by Aaron Schulz:

[mediawiki/core@master] phpunit: Remove redundancy from \"skins\" test and \"extensions\" suites

Reason:

Worked around this in paratest scripts

https://gerrit.wikimedia.org/r/763650

gerritbot added a comment.Mar 7 2022, 9:59 PM

Change 763651 had a related patch set uploaded (by Krinkle; author: Aaron Schulz):

[mediawiki/core@master] tests: Avoid unsafe use of setUpBeforeClass() in ApiFormatXmlTest

https://gerrit.wikimedia.org/r/763651

gerritbot added a comment.Mar 7 2022, 10:01 PM

Change 767232 had a related patch set uploaded (by Krinkle; author: Aaron Schulz):

[mediawiki/core@master] tests: Split out DatabaseSqliteUpgradeTest class

https://gerrit.wikimedia.org/r/767232

gerritbot added a comment.Mar 7 2022, 10:25 PM

Change 763651 merged by jenkins-bot:

[mediawiki/core@master] tests: Avoid unsafe use of setUpBeforeClass() in ApiFormatXmlTest

https://gerrit.wikimedia.org/r/763651

ReleaseTaggerBot added a project: MW-1.38-notes (1.38.0-wmf.25; 2022-03-07).Mar 7 2022, 11:00 PM
gerritbot added a comment.Mar 11 2022, 6:41 PM

Change 767232 merged by jenkins-bot:

[mediawiki/core@master] tests: Split out DatabaseSqliteUpgradeTest class

https://gerrit.wikimedia.org/r/767232

ReleaseTaggerBot edited projects, added MW-1.38-notes (1.38.0-wmf.26; 2022-03-14); removed MW-1.38-notes (1.38.0-wmf.25; 2022-03-07).Mar 11 2022, 7:00 PM
gerritbot added a comment.Apr 4 2022, 6:39 PM

Change 776058 had a related patch set uploaded (by Krinkle; author: Aaron Schulz):

[mediawiki/core@master] phpunit: parse any --boostrap parameter in getopt() call

https://gerrit.wikimedia.org/r/776058

gerritbot added a comment.Apr 4 2022, 7:05 PM

Change 776058 merged by jenkins-bot:

[mediawiki/core@master] phpunit: parse any --boostrap parameter in getopt() call

https://gerrit.wikimedia.org/r/776058

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.6; 2022-04-04).Apr 4 2022, 8:00 PM
hashar removed a subscriber: hashar.Apr 20 2022, 12:39 PM
DAlangi_WMF added a subscriber: DAlangi_WMF.Apr 25 2022, 11:20 PM
gerritbot added a comment.May 25 2022, 6:10 PM

Change 793140 had a related patch set uploaded (by Krinkle; author: Aaron Schulz):

[mediawiki/core@master] tests: Fix memcached test failure with multiple BagOStuff test classes

https://gerrit.wikimedia.org/r/793140

gerritbot added a comment.May 25 2022, 6:30 PM

Change 793140 merged by jenkins-bot:

[mediawiki/core@master] tests: Fix memcached test failure with multiple BagOStuff test classes

https://gerrit.wikimedia.org/r/793140

ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.14; 2022-05-30); removed MW-1.39-notes (1.39.0-wmf.6; 2022-04-04).May 25 2022, 7:00 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL