Page MenuHomePhabricator
Create Task
Maniphest T420857

Skipping in EnWikiFormulaeTest does not work under PHPUnit 10
Closed, ResolvedPublic
Actions

Description

The Math test EnWikiFormulaeTest checks that some files are exists, otherwise the tests are skipped.

But calling markTestSkipped from setUpBeforeClass or via data provider breaks under PHPUnit 10 (phpunit exits).
This needs to be refactored to call markTestSkipped from the test functions and must avoid an empty data provider.

Running with MediaWiki settings because there might be integration tests
PHPUnit 10.5.63 by Sebastian Bergmann and contributors.

Missing test files. Required: tests\phpunit\integration\WikiTexVC/en-wiki-formulae-good.json and tests\phpunit\integration\WikiTexVC/en-wiki-formulae-good-reference.json

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
EnWikiFormulaeTest: Simplify skippingmediawiki/extensions/Mathmaster+10 -27
Customize query in gerrit

Related Objects
Search...

Event Timeline

Umherirrender created this task.Mar 22 2026, 8:37 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 22 2026, 8:37 PM
Umherirrender added a parent task: T328919: Upgrade PHPUnit in MediaWiki code to PHPUnit 10.Mar 22 2026, 8:37 PM
Umherirrender updated the task description. (Show Details)Mar 22 2026, 8:40 PM
Reedy subscribed.Mar 22 2026, 9:59 PM

Looks like ChemRegressionTest in the same situation?

Umherirrender added a comment.Mar 22 2026, 10:18 PM
In T420857#11736398, @Reedy wrote:

Looks like ChemRegressionTest in the same situation?

Skipping from setUp or setUpBeforeClass is okay.
Only the skip from data provider is invalid.

Physikerwelt subscribed.Mar 22 2026, 10:25 PM

It would be even better to find a way to get the files

https://github.com/wikimedia/mediawiki-extensions-Math/blob/master/maintenance/downloadMoreTexVCtests.sh

Since the tests take about 10 minutes to run, I think it would be good to run them in a parallel test job.

gerritbot added a comment.Mar 22 2026, 10:34 PM

Change #1258314 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/Math@master] EnWikiFormulaeTest: Simplify skipping

https://gerrit.wikimedia.org/r/1258314

gerritbot added a project: Patch-For-Review.Mar 22 2026, 10:34 PM
Physikerwelt added a comment.Mar 22 2026, 11:44 PM

I mean we can also remove the tests here. I am moving test generation code to MathSearch in T412658. The tests with all strings used in enwiki (which are very useful indeed to find seldom problems, which are reported as "production error" otherwise) could be moved to MathSearch as well.

gerritbot added a comment.Mon, Apr 27, 8:32 AM

Change #1258314 merged by jenkins-bot:

[mediawiki/extensions/Math@master] EnWikiFormulaeTest: Simplify skipping

https://gerrit.wikimedia.org/r/1258314

ReleaseTaggerBot added a project: MW-1.46-notes (1.46.0-wmf.26; 2026-04-28).Mon, Apr 27, 9:00 AM
Maintenance_bot removed a project: Patch-For-Review.Mon, Apr 27, 9:31 AM
Physikerwelt triaged this task as Medium priority.Mon, Apr 27, 12:52 PM

Screenshot 2026-04-27 at 14.50.03.png (180×1 px, 26 KB)

There is no progress indicator when running the tests. Is there a particular reason why the groups were removed?

Physikerwelt added a comment.EditedMon, Apr 27, 2:45 PM

Overall, the tests don't take significantly longer (now:14min was: approx 9min).

Screenshot 2026-04-27 at 16.43.08.png (228×1 px, 54 KB)

I'm starting a run on GitHub as well, to get more reliable time and energy measurements.

Umherirrender added a comment.Mon, Apr 27, 2:48 PM

After the change there is only one test case for phpunit, that has no progress bar. Before it was more test cases (in chunks of 1000) and each finished test case was shown in the progress bar as a dot.
This is now similiar to ChemRegressionTest.

When it needs support with chunking (and get a progress bar), the skipping part is bit tricky, as the data provider must provide a non-empty array to support PHPUnit 10.

Physikerwelt added a comment.Mon, Apr 27, 2:56 PM
In T420857#11861555, @Umherirrender wrote:

When it needs support with chunking (and get a progress bar), the skipping part is bit tricky, as the data provider must provide a non-empty array to support PHPUnit 10.

I think it's sufficient to change the test name so the user is aware that this test might take up to 30 minutes.

Physikerwelt closed this task as Resolved.Mon, Apr 27, 4:54 PM
Physikerwelt assigned this task to Reedy.

It is actually faster when running it with GitHub compare

https://github.com/physikerwelt/mw-math-testing/actions/runs/25002340272 :

Screenshot 2026-04-27 at 18.51.11.png (660×2 px, 153 KB)

vs

https://github.com/physikerwelt/mw-math-testing/actions/runs/23685728244:

Screenshot 2026-04-27 at 18.52.08.png (664×2 px, 150 KB)

(old)

I also tried to change the test description, but it wasn't easy. So I think we are all good here.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits