Prio Notes:

Some progress notes / a status update.

Running the test job on its own (11200 tests):

$ time composer run --timeout=0 phpunit:entrypoint -- --testsuite extensions --group Database --exclude-group Broken,ParserFuzz,Stub,Standalone
...
real	7m38.242s
user	6m58.434s
sys	0m30.815s

Running tests with classes grouped into 8 groups, but the groups running sequentially (11290 tests):

real	4m15.663s
user	3m44.455s
sys	0m14.330s

With 8 suites running in parallel, and test classes assigned round-robin to suites:

real	2m26.348s
user	8m8.877s
sys	0m29.452s

With 8 suites running parallel, and test classes assigned to balance suite duration:

real	2m5.897s
user	9m47.516s
sys	0m37.622s

In the related subtasks, I've been fixing up some tests that run okay in a normal sequential suite but fail if they are run with an unfamiliar set of classes (or in an unexpected order). I think what confuses me most so far is the difference between the linear run with all tests, and the linear run with the tests split into 8 groups. One obvious theory is that the split groups are running different tests - it's confusing that the number of tests is more in the split run than in the normal run. Another more out-there theory is that in the process of running 11200 tests in the same process, there's enough stored state, leaked memory and overhead to slow the rest of the suite down.

Next steps will be to continue to fix tests that fail on reordering, and to investigate exactly which tests are running in the one configuration that are not in the other.