The current composer-based implementation of PHPUnit parallel test runs collects the tests to run by crawling the filesystem. This means that the ordering of the tests is dependent on the order in which the files were written to disk, which is not stable over different testing environments (for example, a fresh checkout from Zuul vs. a developer's local machine).

This is an issue for us as some tests are not fully isolated from one another in their setup and teardown. The unpredictable order of test runs (which also changes between machines) makes it hard to reproduce and diagnose test run failures where test ordering is responsible for the issue. Having a predictable order will assist us in diagnosing which tests are failing for these reasons and remedy them in the future.

Acceptance Criteria

• The composer phpunit:prepare-parallel:extensions task implements a stable ordering of the test cases within the split groups - for example an alphabetical ordering.