Per the feedback here (T378299#10266092), the way tests are currently assigned to split_groups tends to collect tests from a lot of different extensions into each group because tests are assigned round-robin to their respective buckets. This:
- makes interactions between tests from different extensions more likely
- means that jobs that test a different set of extensions can have a very different test group layout
- more often exposes previously untested combinations of test classes
While in principle detecting more test-state-encapsulation issues is a positive thing, having frequently-occurring cross-extension parallel testing issues to debug makes life harder for developers.
Change the assignment of tests to groups to make the composition of groups more predictable and understandable.
Acceptance Criteria
- The tests are assigned to their split_groups in the order they appear in the phpunit --list-tests-xml output.
Note that as well as making some bugs harder to detect, this might also lead to less balanced times for the split groups (if, for example, a particular extension's tests tend to take longer per class than average).