This patch https://gerrit.wikimedia.org/r/c/integration/quibble/+/758783 proposes to run databaseless, standalone and database group PHPUnit tests concurrently. That seems to work for core and extension tests using MySQL, but there is a failure when run with SQLite. In SQLite, there are 7 errors for SearchEngineTest (https://integration.wikimedia.org/ci/job/integration-quibble-fullrun-sqlite/39/console#console-section-4)
00:07:23.882 There were 2 errors: 00:07:23.882 00:07:23.882 1) SearchEngineTest::testPhraseSearchHighlight 00:07:23.882 Error: Call to a member function getIterator() on null 00:07:23.882 00:07:23.882 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:191 00:07:23.882 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.882 === Logs generated by test case 00:07:23.882 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.882 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.882 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.882 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.882 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.882 === 00:07:23.882 00:07:23.882 2) SearchEngineTest::testAugmentorSearch 00:07:23.882 TypeError: Argument 1 passed to SearchEngine::augmentSearchResults() must implement interface ISearchResultSet, null given, called in /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php on line 341 00:07:23.882 00:07:23.882 /workspace/src/includes/search/SearchEngine.php:829 00:07:23.882 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:341 00:07:23.882 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.882 === Logs generated by test case 00:07:23.882 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.882 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.882 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.882 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.882 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.882 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.882 [localisation] [debug] LocalisationCache::isExpired(en): cache missing, need to make one [] 00:07:23.882 === 00:07:23.882 00:07:23.882 -- 00:07:23.882 00:07:23.882 There were 7 failures: 00:07:23.882 00:07:23.882 1) SearchEngineTest::testFullWidth 00:07:23.882 Failed asserting that null is of type "object". 00:07:23.882 00:07:23.882 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:101 00:07:23.883 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:119 00:07:23.883 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.883 === Logs generated by test case 00:07:23.883 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.883 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.883 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.883 === 00:07:23.883 00:07:23.883 2) SearchEngineTest::testTextSearch 00:07:23.883 Failed asserting that null is of type "object". 00:07:23.883 00:07:23.883 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:101 00:07:23.883 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:138 00:07:23.883 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.883 === Logs generated by test case 00:07:23.883 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.883 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.883 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.883 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.883 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.883 === 00:07:23.883 00:07:23.883 3) SearchEngineTest::testWildcardSearch 00:07:23.883 Failed asserting that null is of type "object". 00:07:23.883 00:07:23.883 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:101 00:07:23.883 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:146 00:07:23.883 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.883 === Logs generated by test case 00:07:23.883 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.883 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.883 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.883 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.883 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.883 === 00:07:23.883 00:07:23.883 4) SearchEngineTest::testPhraseSearch 00:07:23.883 Failed asserting that null is of type "object". 00:07:23.883 00:07:23.883 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:101 00:07:23.883 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:172 00:07:23.883 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.883 === Logs generated by test case 00:07:23.883 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.883 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.883 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.883 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.883 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.883 === 00:07:23.883 00:07:23.883 5) SearchEngineTest::testTextPowerSearch 00:07:23.883 Failed asserting that null is of type "object". 00:07:23.883 00:07:23.883 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:101 00:07:23.884 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:205 00:07:23.884 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.884 === Logs generated by test case 00:07:23.884 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.884 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.884 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.884 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.884 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.884 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.884 [localisation] [debug] LocalisationCache::isExpired(en): cache missing, need to make one [] 00:07:23.884 === 00:07:23.884 00:07:23.884 6) SearchEngineTest::testTitleSearch 00:07:23.884 Failed asserting that null is of type "object". 00:07:23.884 00:07:23.884 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:101 00:07:23.884 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:215 00:07:23.884 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.884 === Logs generated by test case 00:07:23.884 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.884 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.884 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.884 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.884 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.884 === 00:07:23.884 00:07:23.884 7) SearchEngineTest::testTextTitlePowerSearch 00:07:23.884 Failed asserting that null is of type "object". 00:07:23.884 00:07:23.884 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:101 00:07:23.884 /workspace/src/tests/phpunit/includes/search/SearchEngineTest.php:227 00:07:23.884 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:457 00:07:23.884 === Logs generated by test case 00:07:23.884 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.884 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.884 [MessageCache] [debug] MessageCache using store {class} {"class":"HashBagOStuff"} 00:07:23.884 [objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"} 00:07:23.884 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.884 [localisation] [debug] LocalisationCache using store LCStoreNull [] 00:07:23.884 [localisation] [debug] LocalisationCache::isExpired(en): cache missing, need to make one [] 00:07:23.884 === 00:07:23.884 00:07:23.884 ERRORS! 00:07:23.884 Tests: 8961, Assertions: 99559, Errors: 2, Failures: 7, Skipped: 35, Incomplete: 3.
Basically, anything using $this->search->searchText() is getting null back instead of an object. The same errors occur on two runs of this job, so even though this is triggered by the code being run in parallel with another process, it's deterministic in which tests fail.