Infection is a mutation testing framework. Its purpose is to measure the effectiveness of a testsuite, by seeing how it reacts to changes in the code under test. For an overview of how it works, see the manual.

I think it might be interesting to run this for MW, and see how our test suites perform. Note that mutation testing is slow by default: it needs to run the tests multiple times, also with code coverage enabled. Even though infection supports multithreading, we might still consider ways to make the job less impactful, such as restricting it to unit tests, or running it in a daily job.

I played with infection a bit locally, and it seems appealing; however, figuring out how to do a real setup might be somewhat hard. Here are the steps you can follow to try infection on the includes/page directory, with unit tests only.

• Make sure you have a coverage driver available (preferably pcov. If you're using xdebug you're doing it wrong)
• composer require --dev "infection/infection"
• vendor/bin/infection, let the wizard create a minimal config file
• Comment out every fwrite( STDERR call in PHPUnit's bootstrap.php and bootstrap.common.php. For some reason, infection thinks that the test suite failed if stderr isn't empty.
• In PHPUnit's boostrap.php, comment out the whole if ( $GLOBALS['argc'] === 1 ) { conditional, replacing it with $hasIntegrationTests = false;
• Add the following to the autogenerated infection.json5 file: "bootstrap": "./tests/phpunit/bootstrap.php"
• Run vendor/bin/infection --threads=16 --test-framework-options="--testsuite=core:unit" --show-mutations --filter=includes/page/
• Add $this->markTestSkipped() to any unrelated test that's randomly failing for whatever reason
• Repeat the last two steps until the baseline run has no errors
• Profit