Page MenuHomePhabricator
Create Task
Maniphest T395470

Drop deprecated phpunit.php and suite.xml
Closed, ResolvedPublic
Actions

Description

The old tests/phpunit/phpunit.php entrypoint and its associates tests/phpunit/suite.xml configuration file have been deprecated as part of T90875 and some parts were removed via T227900. Developers should now use composer phpunit (or vendor/bin/phpunit directly), and phpunit.xml.dist if they need to do something explicitly with the config file. This task tracks the removal of both, as well associated functionality, from MW core.

To do:

  • Direct usages in CI (codesearch). Some of these might were kept around for REL1_39, where vendor/bin/phpunit was not yet a viable replacement, but that is now EOL.
    • quibble-coverage scripts in integration/config.git
    • Use in integration/quibble.git
      • Dropped
      • Released
      • Uses upgraded
  • Usages in WMF extensions/skins (codesearch)
  • MW core cleanup (codesearch). Also anything referenced by these, like dropping bootstrap.maintenance.php, and merging bootstrap.common.php and bootstrap.integration.php.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
tests: Remove deprecated PHPUnit entry point, config, and bootstrap [take 2]mediawiki/coremaster+10 -350
jjb: Upgrade Quibble jobs to 1.15.0integration/configmaster+32 -32
tests: Remove deprecated PHPUnit entry point, config, and bootstrapmediawiki/coremaster+10 -350
jjb: Upgrade quibble-coverage jobs for fixed direct PHPUnit usageintegration/configmaster+3 -3
Docker: [quibble-coverage] Pass PHPUnit config location explicitlyintegration/configmaster+10 -2
jjb: Add notes about old quibble-coverage pinintegration/configmaster+3 -3
Revert "jjb: Bump quibble-coverage: Direct PHPUnit, include Dump/ParserFuzz/Stub"integration/configmaster+3 -3
jjb: Bump quibble-coverage: Direct PHPUnit, include Dump/ParserFuzz/Stubintegration/configmaster+3 -3
jjb: [mwcore-phpunit-coverage-patch] Invoke PHPUnit directlyintegration/configmaster+1 -1
Docker: [quibble-coverage] Stop using legacy PHPUnit entrypointintegration/configmaster+15 -15
Always use `composer phpunit` for PHPUnit testsintegration/quibblemaster+23 -18
README: Update instructions for running testsmediawiki/extensions/Scoremaster+2 -2
jjb: [mwcore-phpunit-coverage-patch] Stop using legacy PHPUnit entrypointintegration/configmaster+1 -1
tests: add skins to PHPUnit "extensions:unit" test suitemediawiki/coreREL1_44+4 -13
tests: add skins to PHPUnit "extensions:unit" test suitemediawiki/coreREL1_43+4 -13
tests: add skins to PHPUnit "extensions:unit" test suitemediawiki/coreREL1_39+1 -0
tests: add skins to PHPUnit "extensions" test suitemediawiki/coremaster+3 -11
zuul: switch skins coverage to same job as extensionsintegration/configmaster+1 -1
jjb: switch skins coverage to mwext-phpunit-coverageintegration/configmaster+1 -1
quibble-bullseye-php81-coverage: Fix bad `--path-to-mw` argintegration/configmaster+7 -1
quibble-coverage: Fix `--path-to-mw` arg for skinsintegration/configmaster+7 -1
tests: Drop ancient bootstrap.phpmediawiki/extensions/MobileFrontendmaster+0 -18
Update PHPUnit command in run-mediawiki-testsmediawiki/vagrantmaster+1 -1
Update PHPUnit command in Makefile, take 2mediawiki/extensions/UniversalLanguageSelectormaster+1 -1
Update PHPUnit command in Makefile and documentationmediawiki/extensions/CirrusSearchmaster+2 -2
Update PHPUnit command in Makefilemediawiki/extensions/ContentTranslationmaster+2 -2
Update PHPUnit command in Makefilemediawiki/extensions/Elasticamaster+1 -5
Avoid comment reference to deprecated suite.xmlmediawiki/tools/phpunit-patch-coveragemaster+1 -1
Update PHPUnit command in Makefilemediawiki/extensions/GeoDatamaster+1 -1
jjb: Update mwext-*-coverage* jobs to quibble-bullseye-php81-coverage:1.14.2-s2integration/configmaster+3 -3
Update PHPUnit command in Makefilemediawiki/extensions/UniversalLanguageSelectormaster+2 -2
Update PHPUnit command in Makefilemediawiki/extensions/TranslationNotificationsmaster+1 -1
Update PHPUnit command in Makefilemediawiki/extensions/cldrmaster+1 -1
tests: Drop ancient PHPUnit setupmediawiki/extensions/Flowmaster+1 -48
mwext-phpunit-coverage: Switch from legacy phpunit.php to vendor/bin/phpunitintegration/configmaster+16 -12
Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenNoneT225730 Reduce runtime of MW shared gate Jenkins jobs to 5 min
 OpenNoneT50217 Speed up MediaWiki PHPUnit build by running integration tests in parallel
 ResolvedDaimonaT395470 Drop deprecated phpunit.php and suite.xml
 ResolvedReedyT403199 Formally EOL MW 1.39
 ResolvedReedyT404610 Release MediaWiki 1.39.16/1.43.6/1.44.3/1.45.1
Restricted Task
Restricted Task
Restricted Task
 ResolvedSecuritySomeRandomDeveloperT406639 CVE-2025-67477: Stored XSS through a system message in Special:ApiSandbox
Restricted Task
 ResolvedSecuritycscottT407131 CVE-2025-67479: Magic word replacement in legacy parser allows using reserved data attributes through wikitext
 ResolvedSecurityCatropeT401053 CVE-2025-67480: list=allrevisions can be used to bypass Extension:Lockdown
 ResolvedSecuritysbassettT405790 CVE-2025-67481: jQueryMsg style sanitizer regex can be bypassed using character escapes
 ResolvedSecuritymatmarexT251032 CVE-2025-67481: mw.message(…).parse() doesn't output safe HTML, but it's being used as if it does
Restricted Task
Restricted Task
 ResolvedAtienoT401987 Consider deprecating/removing the xslt option from the action api
Restricted Task
Restricted Task
Restricted Task
Restricted Task
Restricted Task
Restricted Task
 ResolvedNoneT404617 Update onwiki release notes for 1.39.16/1.43.6/1.44.3/1.45.1
Restricted Task
 ResolvedZabeT404619 Update HISTORY in master after 1.39.16/1.43.6/1.44.3/1.45.1
 ResolvedmmartoranaT404620 Write and send supplementary release announcement for extensions and skins with security patches (1.39.16/1.43.6/1.44.3/1.45.1)
 ResolvedReedyT412854 Release 1.39.17
 ResolvedDaimonaT90875 Use vendor/bin/phpunit instead of tests/phpunit/phpunit.php
 ResolvedkostajhT297292 Fix "Deprecated: Premature access to service" from data providers when using vendor/bin/phpunit
 ResolvedDaimonaT227900 Phase out usage of tests/phpunit/suite.xml
 ResolveddanielT304183 Allow LocalSettings.php to be loaded in a function scope
 ResolvedBUG REPORTkostajhT306020 PHPUnit integration tests fail with "No such service: HookContainer"

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
gerritbot added a comment.Aug 12 2025, 8:51 PM

Change #1178071 merged by jenkins-bot:

[integration/config@master] quibble-bullseye-php81-coverage: Fix bad `--path-to-mw` arg

https://gerrit.wikimedia.org/r/1178071

gerritbot added a comment.Aug 12 2025, 9:02 PM

Change #1178030 merged by jenkins-bot:

[integration/config@master] jjb: Update mwext-*-coverage* jobs to quibble-bullseye-php81-coverage:1.14.2-s2

https://gerrit.wikimedia.org/r/1178030

Krinkle updated the task description. (Show Details)Aug 12 2025, 9:14 PM
gerritbot added a comment.Aug 12 2025, 9:26 PM

Change #1178042 merged by jenkins-bot:

[mediawiki/extensions/GeoData@master] Update PHPUnit command in Makefile

https://gerrit.wikimedia.org/r/1178042

gerritbot added a comment.Aug 13 2025, 9:49 AM

Change #1178048 merged by jenkins-bot:

[mediawiki/tools/phpunit-patch-coverage@master] Avoid comment reference to deprecated suite.xml

https://gerrit.wikimedia.org/r/1178048

gerritbot added a comment.Aug 13 2025, 10:21 AM

Change #1178044 merged by jenkins-bot:

[mediawiki/extensions/Elastica@master] Update PHPUnit command in Makefile

https://gerrit.wikimedia.org/r/1178044

gerritbot added a comment.Aug 13 2025, 10:21 AM

Change #1178045 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] Update PHPUnit command in Makefile

https://gerrit.wikimedia.org/r/1178045

gerritbot added a comment.Aug 13 2025, 10:21 AM

Change #1178046 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Update PHPUnit command in Makefile and documentation

https://gerrit.wikimedia.org/r/1178046

gerritbot added a comment.Aug 13 2025, 11:21 AM

Change #1178511 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/extensions/UniversalLanguageSelector@master] Update PHPUnit command in Makefile, take 2

https://gerrit.wikimedia.org/r/1178511

gerritbot added a comment.Aug 13 2025, 1:44 PM

Change #1178511 merged by jenkins-bot:

[mediawiki/extensions/UniversalLanguageSelector@master] Update PHPUnit command in Makefile, take 2

https://gerrit.wikimedia.org/r/1178511

gerritbot added a comment.Aug 18 2025, 11:38 PM

Change #1178034 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/extensions/MobileFrontend@master] tests: Drop ancient bootstrap.php

https://gerrit.wikimedia.org/r/1178034

gerritbot added a comment.Aug 18 2025, 11:41 PM

Change #1178047 merged by Daimona Eaytoy:

[mediawiki/vagrant@master] Update PHPUnit command in run-mediawiki-tests

https://gerrit.wikimedia.org/r/1178047

gerritbot added a comment.Aug 19 2025, 3:07 AM

Change #1178034 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] tests: Drop ancient bootstrap.php

https://gerrit.wikimedia.org/r/1178034

ReleaseTaggerBot edited projects, added MW-1.45-notes (1.45.0-wmf.16; 2025-08-26); removed MW-1.45-notes (1.45.0-wmf.15; 2025-08-19).Aug 19 2025, 4:00 AM
Daimona updated the task description. (Show Details)Aug 19 2025, 10:31 AM
gerritbot added a comment.Aug 20 2025, 5:22 PM

Change #1180606 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] quibble-coverage: Fix `--path-to-mw` arg for skins

https://gerrit.wikimedia.org/r/1180606

gerritbot added a comment.Aug 21 2025, 6:38 AM

Change #1180606 abandoned by Hashar:

[integration/config@master] quibble-coverage: Fix `--path-to-mw` arg for skins

Reason:

Per Timo, it is a bit more complicated :)

https://gerrit.wikimedia.org/r/1180606

Jdforrester-WMF added a subtask: T403199: Formally EOL MW 1.39.Aug 28 2025, 5:05 PM
hashar subscribed.Aug 29 2025, 8:24 AM
hashar added a subscriber: Krinkle.Sep 4 2025, 2:10 PM
In T395470#11071207, @gerritbot wrote:

Change #1176721 had a related patch set uploaded (by Krinkle; author: @Krinkle):

[integration/config@master] mwext-phpunit-coverage: Switch from legacy phpunit.php to vendor/bin/phpunit

https://gerrit.wikimedia.org/r/1176721

We need the same patch for the mwskin-phpunit-coverage or alternatively move skins to use the same jobs (mwext-phpunit-coverage and mwext-phpunit-coverage-publish). The later will let us phase out the last mwskin prefixed jobs and fulfill T402398.

gerritbot added a comment.Sep 8 2025, 12:09 PM

Change #1185862 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@master] tests: add skins to PHPUnit "extensions" test suite

https://gerrit.wikimedia.org/r/1185862

gerritbot added a comment.Sep 8 2025, 1:08 PM

Change #1185925 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: switch skins coverage to mwext-phpunit-coverage

https://gerrit.wikimedia.org/r/1185925

gerritbot added a comment.Sep 8 2025, 1:10 PM

Change #1185926 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] zuul: switch skins coverage to same job as extensions

https://gerrit.wikimedia.org/r/1185926

gerritbot added a comment.Sep 8 2025, 2:06 PM

Change #1185862 merged by jenkins-bot:

[mediawiki/core@master] tests: add skins to PHPUnit "extensions" test suite

https://gerrit.wikimedia.org/r/1185862

gerritbot added a comment.Sep 9 2025, 7:30 AM

Change #1185925 merged by jenkins-bot:

[integration/config@master] jjb: switch skins coverage to mwext-phpunit-coverage

https://gerrit.wikimedia.org/r/1185925

gerritbot added a comment.Sep 9 2025, 7:30 AM

Change #1185926 merged by jenkins-bot:

[integration/config@master] zuul: switch skins coverage to same job as extensions

https://gerrit.wikimedia.org/r/1185926

hashar mentioned this in T227900: Phase out usage of tests/phpunit/suite.xml.Sep 9 2025, 8:22 AM
hashar updated the task description. (Show Details)
gerritbot added a comment.Sep 9 2025, 8:37 AM

Change #1186436 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@REL1_39] tests: add skins to PHPUnit "extensions" test suite

https://gerrit.wikimedia.org/r/1186436

gerritbot added a comment.Sep 9 2025, 8:44 AM

Change #1186440 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@REL1_43] tests: add skins to PHPUnit "extensions" test suite

https://gerrit.wikimedia.org/r/1186440

gerritbot added a comment.Sep 9 2025, 8:47 AM

Change #1186441 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@REL1_44] tests: add skins to PHPUnit "extensions" test suite

https://gerrit.wikimedia.org/r/1186441

Daimona added a comment.Sep 12 2025, 12:45 PM

Remaining usages in CI are in quibble-bullseye-php81-coverage and mwcore-phpunit-coverage-patch; presumably both simple updates? Then removing the old syntax from phpunit-suite-edit.py and updating some comments. I think the usages in quibble need to stay for 1.39 but can be cleaned up later this year.

gerritbot added a comment.Sep 17 2025, 12:56 AM

Change #1186436 merged by jenkins-bot:

[mediawiki/core@REL1_39] tests: add skins to PHPUnit "extensions:unit" test suite

https://gerrit.wikimedia.org/r/1186436

gerritbot added a comment.Sep 17 2025, 1:03 AM

Change #1186440 merged by jenkins-bot:

[mediawiki/core@REL1_43] tests: add skins to PHPUnit "extensions:unit" test suite

https://gerrit.wikimedia.org/r/1186440

gerritbot added a comment.Sep 17 2025, 1:03 AM

Change #1186441 merged by jenkins-bot:

[mediawiki/core@REL1_44] tests: add skins to PHPUnit "extensions:unit" test suite

https://gerrit.wikimedia.org/r/1186441

Daimona updated the task description. (Show Details)Oct 7 2025, 12:42 PM
Daimona added a project: User-Daimona.
Reedy mentioned this in T407878: CI noise in mwext-phpunit-coverage-patch.Oct 21 2025, 9:39 PM
Reedy changed the status of subtask T403199: Formally EOL MW 1.39 from Open to In Progress.Dec 16 2025, 6:31 PM
Jdforrester-WMF closed subtask T403199: Formally EOL MW 1.39 as Resolved.Jan 5 2026, 1:40 PM
Daimona updated the task description. (Show Details)Jan 5 2026, 5:47 PM
Jdforrester-WMF added a comment.Jan 5 2026, 6:09 PM

OK, this should now finally be fully unblocked, AIUI?

Daimona added a comment.Jan 5 2026, 7:15 PM
In T395470#11493280, @Jdforrester-WMF wrote:

OK, this should now finally be fully unblocked, AIUI?

Correct. The CI cleanup needs to happen first and I think I wasn't completely sure about a couple places; there's also some extra cleanup like removal of the --path-to-mw option. Then the MW core part should be straightforward.

gerritbot added a comment.Jan 14 2026, 10:38 PM

Change #1226955 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: [mwcore-phpunit-coverage-patch] Switch from legacy PHPUnit entrypoint to composer

https://gerrit.wikimedia.org/r/1226955

gerritbot added a comment.Jan 14 2026, 10:54 PM

Change #1226955 merged by jenkins-bot:

[integration/config@master] jjb: [mwcore-phpunit-coverage-patch] Stop using legacy PHPUnit entrypoint

https://gerrit.wikimedia.org/r/1226955

Jdforrester-WMF added a comment.Jan 14 2026, 10:54 PM
In T395470#11493671, @Daimona wrote:
In T395470#11493280, @Jdforrester-WMF wrote:

OK, this should now finally be fully unblocked, AIUI?

Correct. The CI cleanup needs to happen first and I think I wasn't completely sure about a couple places; there's also some extra cleanup like removal of the --path-to-mw option. Then the MW core part should be straightforward.

I've done the easy one, mwcore-phpunit-coverage-patch, which was defined inline. I'll have a poke at the other four (mwext-phpunit-coverage-patch, mwext-phpunit-coverage, mwext-phpunit-coverage-publish, mwcore-phpunit-coverage-master) later if I have time.

Jdforrester-WMF added subtasks: T90875: Use vendor/bin/phpunit instead of tests/phpunit/phpunit.php, T227900: Phase out usage of tests/phpunit/suite.xml.Jan 14 2026, 10:56 PM
gerritbot added a comment.Jan 21 2026, 10:25 PM

Change #1229739 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[integration/quibble@master] Always use `composer phpunit` for PHPUnit tests

https://gerrit.wikimedia.org/r/1229739

gerritbot added a comment.Jan 21 2026, 10:38 PM

Change #1229756 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[integration/config@master] dockerfiles: Drop legacy PHPUnit entrypoint from quibble-coverage

https://gerrit.wikimedia.org/r/1229756

Daimona added a comment.Jan 21 2026, 10:41 PM
In T395470#11523366, @Jdforrester-WMF wrote:
In T395470#11493671, @Daimona wrote:
In T395470#11493280, @Jdforrester-WMF wrote:

OK, this should now finally be fully unblocked, AIUI?

Correct. The CI cleanup needs to happen first and I think I wasn't completely sure about a couple places; there's also some extra cleanup like removal of the --path-to-mw option. Then the MW core part should be straightforward.

I've done the easy one, mwcore-phpunit-coverage-patch, which was defined inline. I'll have a poke at the other four (mwext-phpunit-coverage-patch, mwext-phpunit-coverage, mwext-phpunit-coverage-publish, mwcore-phpunit-coverage-master) later if I have time.

Thanks! I made patches for quibble and quibble-coverage. It seemed easy enough, but then I don't know if it actually works. Patch for core coming next.

gerritbot added a comment.Jan 21 2026, 10:42 PM

Change #1229764 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/extensions/Score@master] README: Update instructions for running tests

https://gerrit.wikimedia.org/r/1229764

gerritbot added a comment.Jan 21 2026, 10:55 PM

Change #1229777 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/core@master] Remove deprecated PHPUnit entry point, config, and bootstrap

https://gerrit.wikimedia.org/r/1229777

gerritbot added a comment.Jan 22 2026, 12:09 AM

Change #1229764 merged by jenkins-bot:

[mediawiki/extensions/Score@master] README: Update instructions for running tests

https://gerrit.wikimedia.org/r/1229764

Daimona added a comment.Jan 22 2026, 2:30 PM

For mwcore-phpunit-coverage-patch, I just randomly opened a build and noticed that it fails due to the 5m timeout of composer. Switching to the direct invocation should fix that.

gerritbot added a comment.Jan 22 2026, 2:33 PM

Change #1229739 merged by jenkins-bot:

[integration/quibble@master] Always use `composer phpunit` for PHPUnit tests

https://gerrit.wikimedia.org/r/1229739

gerritbot added a comment.Jan 22 2026, 2:33 PM

Change #1230350 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[integration/config@master] jjb: [mwcore-phpunit-coverage-patch] Invoke PHPUnit directly

https://gerrit.wikimedia.org/r/1230350

gerritbot added a comment.Jan 22 2026, 2:45 PM

Change #1230350 merged by jenkins-bot:

[integration/config@master] jjb: [mwcore-phpunit-coverage-patch] Invoke PHPUnit directly

https://gerrit.wikimedia.org/r/1230350

gerritbot added a comment.Jan 22 2026, 5:33 PM

Change #1229756 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-coverage] Stop using legacy PHPUnit entrypoint

https://gerrit.wikimedia.org/r/1229756

Stashbot added a comment.Jan 22 2026, 5:36 PM

Mentioned in SAL (#wikimedia-releng) [2026-01-22T17:36:18Z] <James_F> Docker: [quibble-coverage] Stop using legacy PHPUnit entrypoint (T395470) & Stop excluding Dump/ParserFuzz/Stub groups (T415230)

Stashbot mentioned this in T415230: Remove default PHPUnit exclusion of "Dump", "ParserFuzz", and "Stub" groups.Jan 22 2026, 5:36 PM
gerritbot added a comment.Jan 22 2026, 5:42 PM

Change #1230404 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Bump quibble-coverage: Direct PHPUnit, incclude Dump/ParserFuzz/Stub

https://gerrit.wikimedia.org/r/1230404

Jdforrester-WMF updated the task description. (Show Details)Jan 22 2026, 5:49 PM
gerritbot added a comment.Jan 22 2026, 5:50 PM

Change #1230404 merged by jenkins-bot:

[integration/config@master] jjb: Bump quibble-coverage: Direct PHPUnit, include Dump/ParserFuzz/Stub

https://gerrit.wikimedia.org/r/1230404

Dreamy_Jazz subscribed.EditedJan 23 2026, 1:04 PM

I think this broke mwext-phpunit-coverage-patch @Jdforrester-WMF

I'm getting the following for all CheckUser and ConfirmEdit (CAPTCHA extension) patches (https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-patch/58766/console and https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-patch/58767/console ):

ConfirmEdit example
13:04:13 $ php -d extension=pcov.so -d pcov.enabled=1 -d pcov.directory=/workspace/src/extensions/ConfirmEdit -d pcov.exclude='@(tests|vendor)@' -d pcov.initial.files=3000 "$MW_INSTALL_PATH"/vendor/bin/phpunit --coverage-clover /tmp/clovery7nzdT --filter '/AbuseFilterTest/'
13:04:13 PHPUnit 9.6.21 by Sebastian Bergmann and contributors.
13:04:13 
13:04:13 Usage:
13:04:13   phpunit [options] UnitTest.php
13:04:13   phpunit [options] <directory>
13:04:13 
13:04:13 Code Coverage Options:
13:04:13 
13:04:13   --coverage-clover <file>    Generate code coverage report in Clover XML format
13:04:13   --coverage-cobertura <file> Generate code coverage report in Cobertura XML format
13:04:13   --coverage-crap4j <file>    Generate code coverage report in Crap4J XML format
13:04:13   --coverage-html <dir>       Generate code coverage report in HTML format
13:04:13   --coverage-php <file>       Export PHP_CodeCoverage object to file
13:04:13   --coverage-text=<file>      Generate code coverage report in text format [default: standard output]
13:04:13   --coverage-xml <dir>        Generate code coverage report in PHPUnit XML format
13:04:13   --coverage-cache <dir>      Cache static analysis results
13:04:13   --warm-coverage-cache       Warm static analysis cache
13:04:13   --coverage-filter <dir>     Include <dir> in code coverage analysis
13:04:13   --path-coverage             Perform path coverage analysis
13:04:13   --disable-coverage-ignore   Disable annotations for ignoring code coverage
13:04:13   --no-coverage               Ignore code coverage configuration
13:04:13 
13:04:13 Logging Options:
13:04:13 
13:04:13   --log-junit <file>          Log test execution in JUnit XML format to file
13:04:13   --log-teamcity <file>       Log test execution in TeamCity format to file
13:04:13   --testdox-html <file>       Write agile documentation in HTML format to file
13:04:13   --testdox-text <file>       Write agile documentation in Text format to file
13:04:13   --testdox-xml <file>        Write agile documentation in XML format to file
13:04:13   --reverse-list              Print defects in reverse order
13:04:13   --no-logging                Ignore logging configuration
13:04:13 
13:04:13 Test Selection Options:
13:04:13 
13:04:13   --list-suites               List available test suites
13:04:13   --testsuite <name>          Filter which testsuite to run
13:04:13   --list-groups               List available test groups
13:04:13   --group <name>              Only runs tests from the specified group(s)
13:04:13   --exclude-group <name>      Exclude tests from the specified group(s)
13:04:13   --covers <name>             Only runs tests annotated with "@covers <name>"
13:04:13   --uses <name>               Only runs tests annotated with "@uses <name>"
13:04:13   --list-tests                List available tests
13:04:13   --list-tests-xml <file>     List available tests in XML format
13:04:13   --filter <pattern>          Filter which tests to run
13:04:13   --test-suffix <suffixes>    Only search for test in files with specified suffix(es). Default: Test.php,.phpt
13:04:13 
13:04:13 Test Execution Options:
13:04:13 
13:04:13   --dont-report-useless-tests Do not report tests that do not test anything
13:04:13   --strict-coverage           Be strict about @covers annotation usage
13:04:13   --strict-global-state       Be strict about changes to global state
13:04:13   --disallow-test-output      Be strict about output during tests
13:04:13   --disallow-resource-usage   Be strict about resource usage during small tests
13:04:13   --enforce-time-limit        Enforce time limit based on test size
13:04:13   --default-time-limit <sec>  Timeout in seconds for tests without @small, @medium or @large
13:04:13   --disallow-todo-tests       Disallow @todo-annotated tests
13:04:13 
13:04:13   --process-isolation         Run each test in a separate PHP process
13:04:13   --globals-backup            Backup and restore $GLOBALS for each test
13:04:13   --static-backup             Backup and restore static attributes for each test
13:04:13 
13:04:13   --colors <flag>             Use colors in output ("never", "auto" or "always")
13:04:13   --columns <n>               Number of columns to use for progress output
13:04:13   --columns max               Use maximum number of columns for progress output
13:04:13   --stderr                    Write to STDERR instead of STDOUT
13:04:13   --stop-on-defect            Stop execution upon first not-passed test
13:04:13   --stop-on-error             Stop execution upon first error
13:04:13   --stop-on-failure           Stop execution upon first error or failure
13:04:13   --stop-on-warning           Stop execution upon first warning
13:04:13   --stop-on-risky             Stop execution upon first risky test
13:04:13   --stop-on-skipped           Stop execution upon first skipped test
13:04:13   --stop-on-incomplete        Stop execution upon first incomplete test
13:04:13   --fail-on-incomplete        Treat incomplete tests as failures
13:04:13   --fail-on-risky             Treat risky tests as failures
13:04:13   --fail-on-skipped           Treat skipped tests as failures
13:04:13   --fail-on-warning           Treat tests with warnings as failures
13:04:13   -v|--verbose                Output more verbose information
13:04:13   --debug                     Display debugging information
13:04:13 
13:04:13   --repeat <times>            Runs the test(s) repeatedly
13:04:13   --teamcity                  Report test execution progress in TeamCity format
13:04:13   --testdox                   Report test execution progress in TestDox format
13:04:13   --testdox-group             Only include tests from the specified group(s)
13:04:13   --testdox-exclude-group     Exclude tests from the specified group(s)
13:04:13   --no-interaction            Disable TestDox progress animation
13:04:13   --printer <printer>         TestListener implementation to use
13:04:13 
13:04:13   --order-by <order>          Run tests in order: default|defects|duration|no-depends|random|reverse|size
13:04:13   --random-order-seed <N>     Use a specific random seed <N> for random order
13:04:13   --cache-result              Write test results to cache file
13:04:13   --do-not-cache-result       Do not write test results to cache file
13:04:13 
13:04:13 Configuration Options:
13:04:13 
13:04:13   --prepend <file>            A PHP script that is included as early as possible
13:04:13   --bootstrap <file>          A PHP script that is included before the tests run
13:04:13   -c|--configuration <file>   Read configuration from XML file
13:04:13   --no-configuration          Ignore default configuration file (phpunit.xml)
13:04:13   --extensions <extensions>   A comma separated list of PHPUnit extensions to load
13:04:13   --no-extensions             Do not load PHPUnit extensions
13:04:13   --include-path <path(s)>    Prepend PHP's include_path with given path(s)
13:04:13   -d <key[=value]>            Sets a php.ini value
13:04:13   --cache-result-file <file>  Specify result cache path and filename
13:04:13   --generate-configuration    Generate configuration file with suggested settings
13:04:13   --migrate-configuration     Migrate configuration file to current format
13:04:13 
13:04:13 Miscellaneous Options:
13:04:13 
13:04:13   -h|--help                   Prints this usage information
13:04:13   --version                   Prints the version and exits
13:04:13   --atleast-version <min>     Checks that version is greater than min and exits
13:04:13   --check-version             Checks whether PHPUnit is the latest version and exits
13:04:13 
13:04:13 
13:04:13 In CloverXml.php line 70:
13:04:13                                      
13:04:13   String could not be parsed as XML  
13:04:13                                      
13:04:13 
13:04:13 check [--sha1 [SHA1]] [--test-dir TEST-DIR] [--html [HTML]] [--command COMMAND]
13:04:13 
13:04:13 INFO:quibble.commands:<<< Finish: User commands: mwext-phpunit-coverage-patch, in 0.633 s
CheckUser example
12:56:11 INFO:quibble.commands:>>> Start: User commands: mwext-phpunit-coverage-patch
12:56:11 INFO:quibble.commands:User commands, working directory: /workspace/src
12:56:11 INFO:quibble.commands:mwext-phpunit-coverage-patch
12:56:11 WARNING:backend.ChromeWebDriver:[1769172971.434][SEVERE]: bind() failed: Cannot assign requested address (99)
12:56:11 + EXT_DIR=/workspace/src/extensions/CheckUser
12:56:11 + cd /workspace/src/extensions/CheckUser
12:56:11 + TEST_DIR=tests/phpunit
12:56:11 + '[' '!' -d tests/phpunit ']'
12:56:11 + phpunit-suite-edit /workspace/src/phpunit.xml.dist --cover-extension extensions/CheckUser
12:56:11 + exec phpunit-patch-coverage check --command 'php -d extension=pcov.so -d pcov.enabled=1 -d pcov.directory=/workspace/src/extensions/CheckUser -d pcov.exclude='\''@(tests|vendor)@'\'' -d pcov.initial.files=3000 "$MW_INSTALL_PATH"/vendor/bin/phpunit' --html /workspace/log/coverage.html --test-dir tests/phpunit
12:56:11 Finding coverage difference in 8d15495672cca3a2e6b231488614afabbb65391d
12:56:11 $ php -d extension=pcov.so -d pcov.enabled=1 -d pcov.directory=/workspace/src/extensions/CheckUser -d pcov.exclude='@(tests|vendor)@' -d pcov.initial.files=3000 "$MW_INSTALL_PATH"/vendor/bin/phpunit --coverage-clover /tmp/cloverMJbhx1 --filter '/RLRegisterModulesHandlerTest|RLRegisterModulesHandlerTest|SpecialSuggestedInvestigationsTest/'
12:56:11 PHPUnit 9.6.21 by Sebastian Bergmann and contributors.
12:56:11 
12:56:11 Usage:
12:56:11   phpunit [options] UnitTest.php
12:56:11   phpunit [options] <directory>
12:56:11 
12:56:11 Code Coverage Options:
12:56:11 
12:56:11   --coverage-clover <file>    Generate code coverage report in Clover XML format
12:56:11   --coverage-cobertura <file> Generate code coverage report in Cobertura XML format
12:56:11   --coverage-crap4j <file>    Generate code coverage report in Crap4J XML format
12:56:11   --coverage-html <dir>       Generate code coverage report in HTML format
12:56:11   --coverage-php <file>       Export PHP_CodeCoverage object to file
12:56:11   --coverage-text=<file>      Generate code coverage report in text format [default: standard output]
12:56:11   --coverage-xml <dir>        Generate code coverage report in PHPUnit XML format
12:56:11   --coverage-cache <dir>      Cache static analysis results
12:56:11   --warm-coverage-cache       Warm static analysis cache
12:56:11   --coverage-filter <dir>     Include <dir> in code coverage analysis
12:56:11   --path-coverage             Perform path coverage analysis
12:56:11   --disable-coverage-ignore   Disable annotations for ignoring code coverage
12:56:11   --no-coverage               Ignore code coverage configuration
12:56:11 
12:56:11 Logging Options:
12:56:11 
12:56:11   --log-junit <file>          Log test execution in JUnit XML format to file
12:56:11   --log-teamcity <file>       Log test execution in TeamCity format to file
12:56:11   --testdox-html <file>       Write agile documentation in HTML format to file
12:56:11   --testdox-text <file>       Write agile documentation in Text format to file
12:56:11   --testdox-xml <file>        Write agile documentation in XML format to file
12:56:11   --reverse-list              Print defects in reverse order
12:56:11   --no-logging                Ignore logging configuration
12:56:11 
12:56:11 Test Selection Options:
12:56:11 
12:56:11   --list-suites               List available test suites
12:56:11   --testsuite <name>          Filter which testsuite to run
12:56:11   --list-groups               List available test groups
12:56:11   --group <name>              Only runs tests from the specified group(s)
12:56:11   --exclude-group <name>      Exclude tests from the specified group(s)
12:56:11   --covers <name>             Only runs tests annotated with "@covers <name>"
12:56:11   --uses <name>               Only runs tests annotated with "@uses <name>"
12:56:11   --list-tests                List available tests
12:56:11   --list-tests-xml <file>     List available tests in XML format
12:56:11   --filter <pattern>          Filter which tests to run
12:56:11   --test-suffix <suffixes>    Only search for test in files with specified suffix(es). Default: Test.php,.phpt
12:56:11 
12:56:11 Test Execution Options:
12:56:11 
12:56:11   --dont-report-useless-tests Do not report tests that do not test anything
12:56:11   --strict-coverage           Be strict about @covers annotation usage
12:56:11   --strict-global-state       Be strict about changes to global state
12:56:11   --disallow-test-output      Be strict about output during tests
12:56:11   --disallow-resource-usage   Be strict about resource usage during small tests
12:56:11   --enforce-time-limit        Enforce time limit based on test size
12:56:11   --default-time-limit <sec>  Timeout in seconds for tests without @small, @medium or @large
12:56:11   --disallow-todo-tests       Disallow @todo-annotated tests
12:56:11 
12:56:11   --process-isolation         Run each test in a separate PHP process
12:56:11   --globals-backup            Backup and restore $GLOBALS for each test
12:56:11   --static-backup             Backup and restore static attributes for each test
12:56:11 
12:56:11   --colors <flag>             Use colors in output ("never", "auto" or "always")
12:56:11   --columns <n>               Number of columns to use for progress output
12:56:11   --columns max               Use maximum number of columns for progress output
12:56:11   --stderr                    Write to STDERR instead of STDOUT
12:56:11   --stop-on-defect            Stop execution upon first not-passed test
12:56:11   --stop-on-error             Stop execution upon first error
12:56:11   --stop-on-failure           Stop execution upon first error or failure
12:56:11   --stop-on-warning           Stop execution upon first warning
12:56:11   --stop-on-risky             Stop execution upon first risky test
12:56:11   --stop-on-skipped           Stop execution upon first skipped test
12:56:11   --stop-on-incomplete        Stop execution upon first incomplete test
12:56:11   --fail-on-incomplete        Treat incomplete tests as failures
12:56:11   --fail-on-risky             Treat risky tests as failures
12:56:11   --fail-on-skipped           Treat skipped tests as failures
12:56:11   --fail-on-warning           Treat tests with warnings as failures
12:56:11   -v|--verbose                Output more verbose information
12:56:11   --debug                     Display debugging information
12:56:11 
12:56:11   --repeat <times>            Runs the test(s) repeatedly
12:56:11   --teamcity                  Report test execution progress in TeamCity format
12:56:11   --testdox                   Report test execution progress in TestDox format
12:56:11   --testdox-group             Only include tests from the specified group(s)
12:56:11   --testdox-exclude-group     Exclude tests from the specified group(s)
12:56:11   --no-interaction            Disable TestDox progress animation
12:56:11   --printer <printer>         TestListener implementation to use
12:56:11 
12:56:11   --order-by <order>          Run tests in order: default|defects|duration|no-depends|random|reverse|size
12:56:11   --random-order-seed <N>     Use a specific random seed <N> for random order
12:56:11   --cache-result              Write test results to cache file
12:56:11   --do-not-cache-result       Do not write test results to cache file
12:56:11 
12:56:11 Configuration Options:
12:56:11 
12:56:11   --prepend <file>            A PHP script that is included as early as possible
12:56:11   --bootstrap <file>          A PHP script that is included before the tests run
12:56:11   -c|--configuration <file>   Read configuration from XML file
12:56:11   --no-configuration          Ignore default configuration file (phpunit.xml)
12:56:11   --extensions <extensions>   A comma separated list of PHPUnit extensions to load
12:56:11   --no-extensions             Do not load PHPUnit extensions
12:56:11   --include-path <path(s)>    Prepend PHP's include_path with given path(s)
12:56:11   -d <key[=value]>            Sets a php.ini value
12:56:11   --cache-result-file <file>  Specify result cache path and filename
12:56:11   --generate-configuration    Generate configuration file with suggested settings
12:56:11   --migrate-configuration     Migrate configuration file to current format
12:56:11 
12:56:11 Miscellaneous Options:
12:56:11 
12:56:11   -h|--help                   Prints this usage information
12:56:11   --version                   Prints the version and exits
12:56:11   --atleast-version <min>     Checks that version is greater than min and exits
12:56:11   --check-version             Checks whether PHPUnit is the latest version and exits
12:56:11 
12:56:11 
12:56:11 In CloverXml.php line 70:
12:56:11                                      
12:56:11   String could not be parsed as XML  
12:56:11                                      
12:56:11 
12:56:11 check [--sha1 [SHA1]] [--test-dir TEST-DIR] [--html [HTML]] [--command COMMAND]
12:56:11 
12:56:11 INFO:quibble.commands:<<< Finish: User commands: mwext-phpunit-coverage-patch, in 0.556 s
Daimona added a comment.Jan 23 2026, 2:53 PM
In T395470#11548714, @Dreamy_Jazz wrote:

I think this broke mwext-phpunit-coverage-patch

I tested this locally, I'm getting a slightly different failure mode where PHPUnit prints its usage info and exits, without a specific error message (whereas in CI there's something about invalid XML). My initial guess was that this is running the command in the extension directory and not using the phpunit.xml.dist file in MW core. This seems to be confirmed because locally it starts working if I manually specify the config file with -c.

Not sure what the best way to fix this is, whether to always run PHPUnit in the core directory, or specifying the config file with -c, or having phpunit-suite-edit create a config copy in the current directory. I can see pros and cons to each approach and I'm not sure which one I prefer off the top of my head.

gerritbot added a comment.Jan 23 2026, 2:56 PM

Change #1230930 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Revert "jjb: Bump quibble-coverage: Direct PHPUnit, include Dump/ParserFuzz/Stub"

https://gerrit.wikimedia.org/r/1230930

gerritbot added a comment.Jan 23 2026, 3:00 PM

Change #1230930 merged by jenkins-bot:

[integration/config@master] Revert "jjb: Bump quibble-coverage: Direct PHPUnit, include Dump/ParserFuzz/Stub"

https://gerrit.wikimedia.org/r/1230930

Jdforrester-WMF added a comment.Jan 23 2026, 3:03 PM

I've temporarily reverted the change.

gerritbot added a comment.Jan 24 2026, 1:29 AM

Change #1229777 merged by jenkins-bot:

[mediawiki/core@master] tests: Remove deprecated PHPUnit entry point, config, and bootstrap

https://gerrit.wikimedia.org/r/1229777

Reedy mentioned this in T415455: mwcore-phpunit-coverage-master failing due to Could not open input file: tests/phpunit/phpunit.php.Jan 25 2026, 3:07 AM
Reedy mentioned this in T415471: mwext-phpunit-coverage-patch failing due to FileNotFoundError: [Errno 2] No such file or directory: '/workspace/src/tests/phpunit/suite.xml'.Jan 25 2026, 8:19 PM
gerritbot added a comment.Jan 26 2026, 1:59 PM

Change #1233194 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Add notes about old quibble-coverage pin

https://gerrit.wikimedia.org/r/1233194

gerritbot added a comment.Jan 26 2026, 2:04 PM

Change #1233194 merged by jenkins-bot:

[integration/config@master] jjb: Add notes about old quibble-coverage pin

https://gerrit.wikimedia.org/r/1233194

Jdforrester-WMF updated the task description. (Show Details)Jan 26 2026, 2:09 PM
Daimona added a comment.Jan 26 2026, 4:20 PM

Having thought about this a bit more:

In T395470#11549121, @Daimona wrote:

always run PHPUnit in the core directory

This might be too easy to miss, or accidentally break.

or specifying the config file with -c

I think this would be the simplest solution, and pretty effective/obvious. The one thing I don't like is the hardcoding of the config file name in more places, which is in the opposite spirit as providing a one-keyword composer entrypoint.

or having phpunit-suite-edit create a config copy in the current directory

This is the approach I like the most in principle. We could have it use phpunit.xml (no .dist), so that the file gets always used automagically: PHPUnit tries phpunit.xml before phpunit.xml.dist; and since the tool is for CI only, I don't think there'd be issues in overriding an already existent phpunit.xml file in the current directory. Alternatively, we could name the file something unique, e.g. phpunit-suite-edit-config.xml and pass that to PHPUnit via -c. It's maybe a bit cleaner and explicit, but there's more code involved (still adding -c to the invocations), and I'm not sure if we need it.

The one thing to watch out for with this approach is that we wanna make sure we don't mess up the paths: all paths in core's phpunit.xml.dist are relative to MW core, but would need to be adapted if we're copying the file to a different directory. This includes test locations, source location (for coverage itself), and at least the bootstrap file location.

All in all, while I'd rather see the third approach, it also seems a somewhat risky change and I wouldn't wanna see another revert just yet. So perhaps we can go with the second approach, for the time being. I'll make a patch.

gerritbot added a comment.Jan 26 2026, 4:27 PM

Change #1233211 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[integration/config@master] Docker: [quibble-coverage] Pass PHPUnit config location explicitly

https://gerrit.wikimedia.org/r/1233211

gerritbot added a comment.Jan 27 2026, 3:12 PM

Change #1233211 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-coverage] Pass PHPUnit config location explicitly

https://gerrit.wikimedia.org/r/1233211

Stashbot added a comment.Jan 27 2026, 3:12 PM

Mentioned in SAL (#wikimedia-releng) [2026-01-27T15:12:41Z] <James_F> Docker: [quibble-coverage] Pass PHPUnit config location explicitly, for T395470

gerritbot added a comment.Jan 27 2026, 3:25 PM

Change #1233739 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Upgrade quibble-coverage jobs for fixed direct PHPUnit usage

https://gerrit.wikimedia.org/r/1233739

Jdforrester-WMF updated the task description. (Show Details)Jan 27 2026, 3:26 PM
gerritbot added a comment.Jan 27 2026, 3:27 PM

Change #1233739 merged by jenkins-bot:

[integration/config@master] jjb: Upgrade quibble-coverage jobs for fixed direct PHPUnit usage

https://gerrit.wikimedia.org/r/1233739

gerritbot added a comment.Jan 27 2026, 3:40 PM

Change #1233741 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] tests: Remove deprecated PHPUnit entry point, config, and bootstrap [take 2]

https://gerrit.wikimedia.org/r/1233741

gerritbot added a comment.Jan 27 2026, 4:04 PM

Change #1233744 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Upgrade Quibble jobs to 1.15.0

https://gerrit.wikimedia.org/r/1233744

Jdforrester-WMF updated the task description. (Show Details)Jan 27 2026, 4:15 PM
gerritbot added a comment.Jan 27 2026, 4:18 PM

Change #1233744 merged by jenkins-bot:

[integration/config@master] jjb: Upgrade Quibble jobs to 1.15.0

https://gerrit.wikimedia.org/r/1233744

gerritbot added a comment.Jan 27 2026, 4:54 PM

Change #1233741 merged by jenkins-bot:

[mediawiki/core@master] tests: Remove deprecated PHPUnit entry point, config, and bootstrap [take 2]

https://gerrit.wikimedia.org/r/1233741

Jdforrester-WMF updated the task description. (Show Details)Jan 27 2026, 4:57 PM
Jdforrester-WMF added a comment.Jan 27 2026, 5:00 PM

Is there anything left to do here? Documentation to update? Otherwise, it's wonderful to get to this point.

ReleaseTaggerBot added a project: MW-1.46-notes (1.46.0-wmf.14; 2026-02-03).Jan 27 2026, 5:00 PM
Daimona closed this task as Resolved.Jan 27 2026, 10:11 PM
Daimona claimed this task.

Nope, everything should've been taken care of already in the last couple years. 🎉

jgleeson mentioned this in T419890: Update fundraising-dev phpunit script.Mar 12 2026, 6:25 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits