Debugging when running unit tests in parallel is one of the most challenging tasks. In our current PHPUnit test setup, the test splitter mixes tests from all projects (Core and gated extensions) into split groups for load balancing. The logged information does not indicate which repo the running tests in each split group come from. This makes it challenging to know exactly which tests are running in each group, and in particular, makes it difficult to debug the tests. Sample logs are attached below (Full output at P90599), and as indicated, it is difficult to distinguish which repo the tests running in each split group belong to:
13:22:06 INFO:quibble.commands:>>> Start: PHPUnit extensions suite (without database or standalone) parallel run (Composer) 13:22:07 > MediaWiki\Composer\ComposerLaunchParallel::launchTestsDatabaseless 13:22:07 Running command ''composer' 'run' '--timeout=0' 'phpunit:entrypoint' '--' '--configuration' '/workspace/src/phpunit-databaseless.xml' '--testsuite' 'split_group_0' '--exclude-group' 'Broken,Standalone,Database' '--cache-result-file=/workspace/log/phpunit_group_0_databaseless.result.cache'' ... 13:22:07 Running command ''composer' 'run' '--timeout=0' 'phpunit:entrypoint' '--' '--configuration' '/workspace/src/phpunit-databaseless.xml' '--testsuite' 'split_group_1' '--exclude-group' 'Broken,Standalone,Database' '--cache-result-file=/workspace/log/phpunit_group_1_databaseless.result.cache'' ... 13:22:07 Running command ''composer' 'run' '--timeout=0' 'phpunit:entrypoint' '--' '--configuration' '/workspace/src/phpunit-databaseless.xml' '--testsuite' 'split_group_4' '--exclude-group' 'Broken,Standalone,Database' '--cache-result-file=/workspace/log/phpunit_group_4_databaseless.result.cache'' ... 13:22:07 Running command ''composer' 'run' '--timeout=0' 'phpunit:entrypoint' '--' '--configuration' '/workspace/src/phpunit-databaseless.xml' '--testsuite' 'split_group_2' '--exclude-group' 'Broken,Standalone,Database' '--cache-result-file=/workspace/log/phpunit_group_2_databaseless.result.cache'' ... 13:22:07 Running command ''composer' 'run' '--timeout=0' 'phpunit:entrypoint' '--' '--configuration' '/workspace/src/phpunit-databaseless.xml' '--testsuite' 'split_group_5' '--exclude-group' 'Broken,Standalone,Database' '--cache-result-file=/workspace/log/phpunit_group_5_databaseless.result.cache'' ... 13:22:07 Running command ''composer' 'run' '--timeout=0' 'phpunit:entrypoint' '--' '--configuration' '/workspace/src/phpunit-databaseless.xml' '--testsuite' 'split_group_3' '--exclude-group' 'Broken,Standalone,Database' '--cache-result-file=/workspace/log/phpunit_group_3_databaseless.result.cache'' ... 13:22:07 Running command ''composer' 'run' '--timeout=0' 'phpunit:entrypoint' '--' '--configuration' '/workspace/src/phpunit-databaseless.xml' '--testsuite' 'split_group_6' '--exclude-group' 'Broken,Standalone,Database' '--cache-result-file=/workspace/log/phpunit_group_6_databaseless.result.cache'' ... 13:22:15 > Composer\Config::disableProcessTimeout 13:22:15 > @php tests/phpunit/generatePHPUnitConfig.php '--configuration' '/workspace/src/phpunit-databaseless.xml' '--testsuite' 'split_group_3' '--exclude-group' 'Broken,Standalone,Database' '--cache-result-file=/workspace/log/phpunit_group_3_databaseless.result.cache'
Acceptance Criteria:
- Check if there's more information we can log from Quible when running unit tests.
- If there is more information to be logged, create a patch.