Page MenuHomePhabricator

PHPUnit: Allow test runner to use main wiki database
Open, Needs TriagePublic

Description

In MediaWikiIntegrationTestCaes and in ParserTestRunner, we have code that clones the main database structure, and runs setup/teardown code before/after each class so that each class has a clean slate for its tests.

That is nice for good but probably is unnecessary in a fair number of tests that access the database, and adds a significant performance cost to integration tests.

Profiling a request shows that cloning the table structure is 30 percent of the time needed to execute a class with a single test:

 php -d xdebug.mode=profile tests/phpunit/phpunit.php extensions/GrowthExperiments/tests/phpunit/integration/SpecialClaimMenteeTest.php
Using PHP 7.4.26
PHPUnit 8.5.21 by Sebastian Bergmann and contributors.

..                                                                  2 / 2 (100%)

Time: 2.4 seconds, Memory: 61.00 MB

OK (2 tests, 2 assertions)

image.png (2×1 px, 1 MB)

In contrast, not using the clone database code cuts the execution time in half:

 PHPUNIT_USE_MAIN_WIKI_DB=1 php -d xdebug.mode=profile tests/phpunit/phpunit.php extensions/GrowthExperiments/tests/phpunit/integration/SpecialClaimMenteeTest.php
Using PHP 7.4.26
PHPUnit 8.5.21 by Sebastian Bergmann and contributors.

..                                                                  2 / 2 (100%)

Time: 1.12 seconds, Memory: 61.00 MB

OK (2 tests, 2 assertions)


You should really speed up these slow tests (>50ms)...
 1. 562ms to run GrowthExperiments\Tests\SpecialClaimMenteeTest:testNonMentorCantExecute

image.png (2×1 px, 1 MB)

It would be nice to have an option (for CI especially) to use the main wiki database unless a test class specifically requests that a cloned database is used.

Event Timeline

Change 745468 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] phpunit: Add option to use main wiki DB

https://gerrit.wikimedia.org/r/745468

Change 745467 had a related patch set uploaded (by Krinkle; author: Kosta Harlan):

[mediawiki/core@master] phpunit: Initialize CLI options for vendor/bin/phpunit

https://gerrit.wikimedia.org/r/745467

Change 745467 merged by jenkins-bot:

[mediawiki/core@master] phpunit: Initialize CLI options for vendor/bin/phpunit

https://gerrit.wikimedia.org/r/745467

Change 745468 abandoned by Kosta Harlan:

[mediawiki/core@master] [WIP] phpunit: Add option to use main wiki DB

Reason:

https://gerrit.wikimedia.org/r/745468