Page MenuHomePhabricator

Reproducible tests with mediawiki-docker (for maths)
Open, Needs TriagePublic

Description

I wrote the a script for reproducible tests runs with MediaWiki docker.

While it installs a fresh version of MediaWiki (03e74942bd9777be96f2a29d7f169acacd9c1176) and the corresponding extension Math (81dedc19091274d36b13c6a9a09f6161fe2e9483) it fails:

> phpunit './extensions/Math/tests/phpunit/'
Using PHP 8.1.20
Running with MediaWiki settings because there might be integration tests
PHP Warning:  Undefined array key "\intent{a}{b}" in /var/www/html/w/extensions/Math/tests/phpunit/unit/WikiTexVC/MMLGenerationTexUtilTest.php on line 249
PHPUnit 9.6.16 by Sebastian Bergmann and contributors.

................................................SSSSSSSSSSSSS   61 / 2067 (  2%)
........SSS..................................................  122 / 2067 (  5%)
....EEEEESSSSSS........................................S...EE  183 / 2067 (  8%)
EEEEEEEEEEEEE.EEEEEE.........................................  244 / 2067 ( 11%)
....................SSPHP Deprecated:  $wgUser reassignment detected [Called from PHPUnit\Framework\TestSuite::run in /var/www/html/w/vendor/phpunit/phpunit/src/Framework/TestSuite.php at line 628] in /var/www/html/w/includes/debug/MWDebug.php on line 379
SS.....................................  305 / 2067 ( 14%)
.............................................................  366 / 2067 ( 17%)
.............................................................  427 / 2067 ( 20%)
.............................................................  488 / 2067 ( 23%)
.............................................................  549 / 2067 ( 26%)
.............................................................  610 / 2067 ( 29%)
.............................................................  671 / 2067 ( 32%)
.............................................................  732 / 2067 ( 35%)
.............................................................  793 / 2067 ( 38%)
.............................................................  854 / 2067 ( 41%)
.............................................................  915 / 2067 ( 44%)
.............................................................  976 / 2067 ( 47%)
............................................................. 1037 / 2067 ( 50%)
............................................................. 1098 / 2067 ( 53%)
............................................................. 1159 / 2067 ( 56%)
............................................................. 1220 / 2067 ( 59%)
............................................................. 1281 / 2067 ( 61%)
............................................................. 1342 / 2067 ( 64%)
............................................................. 1403 / 2067 ( 67%)
............................................................. 1464 / 2067 ( 70%)
............................................................. 1525 / 2067 ( 73%)
............................................................. 1586 / 2067 ( 76%)
............................................................. 1647 / 2067 ( 79%)
............................................................. 1708 / 2067 ( 82%)
............................................................. 1769 / 2067 ( 85%)
............................................................. 1830 / 2067 ( 88%)
............................................................. 1891 / 2067 ( 91%)
............................................................. 1952 / 2067 ( 94%)
............................................................. 2013 / 2067 ( 97%)
......................................................        2067 / 2067 (100%)

You should really speed up these slow tests (>100ms)...
 1. 127ms to run MediaWiki\\Extension\\Math\\Tests\\WikiTexVC\\AllTest::testRunCases with data set "Big"
 2. 110ms to run MediaWiki\\Extension\\Math\\Tests\\PreferencesIntegrationTest::testMathOptionRegistered
Key "JobTypeConf" not found in input sources
Script phpunit handling the phpunit event returned with error code 2
Script @phpunit was called via phpunit:entrypoint

When I check out the proposed fix for

PHP Warning: Undefined array key "\intent{a}{b}" in /var/www/html/w/extensions/Math/tests/phpunit/unit/WikiTexVC/MMLGenerationTexUtilTest.php on line 249

(07a41967ce09c4ed47a6332d4140467bbe298180) the line disappears but the other problems are still present.