Change 704936 merged by jenkins-bot:

[integration/quibble@master] Release Quibble 1.0.0

https://gerrit.wikimedia.org/r/704936

Mentioned in SAL (#wikimedia-releng) [2021-07-16T17:12:11Z] <hashar> Tag Quibble 1.0.0 @ a13d133f7d1 # T286187 T280506 T90875 T218534 T227352

Change 702823 abandoned by Umherirrender:

[mediawiki/core@REL1_31] Add a composer command for the custom PHPUnit entry point

Reason:

This release is EOL

https://gerrit.wikimedia.org/r/702823

Change 745264 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] phpunit: Remove phpunit.php wrapper script

https://gerrit.wikimedia.org/r/745264

Change 745467 had a related patch set uploaded (by Krinkle; author: Kosta Harlan):

[mediawiki/core@master] phpunit: Initialize CLI options for vendor/bin/phpunit

https://gerrit.wikimedia.org/r/745467

Change 745467 merged by jenkins-bot:

[mediawiki/core@master] phpunit: Initialize CLI options for vendor/bin/phpunit

https://gerrit.wikimedia.org/r/745467

Change 702822 merged by jenkins-bot:

[mediawiki/core@REL1_35] Add a composer command for the custom PHPUnit entry point

https://gerrit.wikimedia.org/r/702822

Change 702821 merged by jenkins-bot:

[mediawiki/core@REL1_36] Add a composer command for the custom PHPUnit entry point

https://gerrit.wikimedia.org/r/702821

Change 760301 had a related patch set uploaded (by Krinkle; author: Tim Starling):

[mediawiki/core@master] Improve ObjectCache integration tests

https://gerrit.wikimedia.org/r/760301

Change 760301 merged by jenkins-bot:

[mediawiki/core@master] Improve ObjectCache integration tests

https://gerrit.wikimedia.org/r/760301

Change 775834 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/core@master] Kill MediaWikiPHPUnitCommand

https://gerrit.wikimedia.org/r/775834

Change 775854 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/core@master] Remove some unneeded stuff from phpunit.php and clean up

https://gerrit.wikimedia.org/r/775854

Change 775834 merged by jenkins-bot:

[mediawiki/core@master] phpunit: Remove custom MediaWikiPHPUnitCommand and use default directly

https://gerrit.wikimedia.org/r/775834

Change 775854 merged by jenkins-bot:

[mediawiki/core@master] phpunit: Remove or inline phpunit.php loadSettings() and fatalError()

https://gerrit.wikimedia.org/r/775854

Change 785367 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/core@master] Remove some unnecessary stuff from phpunit.php

https://gerrit.wikimedia.org/r/785367

Change 785367 merged by jenkins-bot:

[mediawiki/core@master] phpunit: Remove some unnecessary code from phpunit.php

https://gerrit.wikimedia.org/r/785367

Change 785383 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/core@master] Move wfRequireOnceInGlobalScope to TestSetup

https://gerrit.wikimedia.org/r/785383

Change 785383 merged by jenkins-bot:

[mediawiki/core@master] Move wfRequireOnceInGlobalScope to TestSetup

https://gerrit.wikimedia.org/r/785383

Change 741970 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] phpunit: Use vendor/bin/phpunit and memove suite.xml

https://gerrit.wikimedia.org/r/741970

Change 745264 abandoned by Kosta Harlan:

[mediawiki/core@master] phpunit: Remove phpunit.php wrapper script

Reason:

squashed

https://gerrit.wikimedia.org/r/745264

Change 802153 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/Wikibase@master] Wikibase.example.php: Use global keyword

https://gerrit.wikimedia.org/r/802153

Change 802154 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/quibble@master] composer: Set default memory limit to unlimited

https://gerrit.wikimedia.org/r/802154

Change 802154 abandoned by Kosta Harlan:

[integration/quibble@master] composer: Set default memory limit to unlimited

Reason:

https://gerrit.wikimedia.org/r/802154

Change 802184 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/Flow@master] ConfirmEditTest: Override Parser service

https://gerrit.wikimedia.org/r/802184

Change 802201 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/Wikibase@master] SetClaimTest: Mock API request to siteinfo

https://gerrit.wikimedia.org/r/802201

Change 802184 merged by jenkins-bot:

[mediawiki/extensions/Flow@master] ConfirmEditTest: Override Parser service

https://gerrit.wikimedia.org/r/802184

Change 802153 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Wikibase.example.php: Use global keyword

https://gerrit.wikimedia.org/r/802153

Change 803487 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] Remove hardcoded references to suite.xml and phpunit.php

https://gerrit.wikimedia.org/r/803487

Change 803525 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] jjb: Use composer phpunit:entrypoint

https://gerrit.wikimedia.org/r/803525

Change 802201 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] SetClaimTest: Mock API request to siteinfo

https://gerrit.wikimedia.org/r/802201

Change 741970 merged by jenkins-bot:

[mediawiki/core@master] phpunit: Default to vendor/bin/phpunit, remove suites.xml

https://gerrit.wikimedia.org/r/741970

For the record, this task is not yet done as we still need to update CI jobs. Some of which are currently failing (e.g. the periodic coverage job).

Still pending:

In T90875#7985550, @gerritbot wrote:

Change 803487 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] Remove hardcoded references to suite.xml and phpunit.php

https://gerrit.wikimedia.org/r/803487

One thing I noticed with the new system, in particular the new bootstrap, is that it loads every extension and skin in the respective directories. This is not ideal for me, because I have some duplicated extension folders (e.g. uncommitted WIPs), some extensions are not updated (because I don't use them), and in general, I fear that loading all extensions could slow everything down. I'm not sure how this can be fixed though, because I assume we don't want to load LocalSettings.php...

@Daimona I'm not aware of this intentionally having changed. If you use composer phpunit, that should use the same maintenance (now "integration") bootstrap for PHPUnit as before, including in fact the loading of LocalSettings.php.

It's only the pure-unit tests composer phpunit:unit which runs core-only tests without LocalSettings and (with DB/servicewiring disallowed, naturally), and I believe thee's a composer phpunit:extensions variant as well which allows the same thing in CI for extensions' pure unit tests. Afaik we don't have to use these locally though.

In T90875#7990832, @Krinkle wrote:

@Daimona I'm not aware of this intentionally having changed. If you use composer phpunit, that should use the same maintenance (now "integration") bootstrap for PHPUnit as before, including in fact the loading of LocalSettings.php.
It's only the pure-unit tests composer phpunit:unit which runs core-only tests without LocalSettings and (with DB/servicewiring disallowed, naturally), and I believe thee's a composer phpunit:extensions variant as well which allows the same thing in CI for extensions' pure unit tests. Afaik we don't have to use these locally though.

I run tests from PHPStorm (using the phpunit executable), and faced this problem when running a unit test. I was already using the same setup before and never run into this issue.

Ack. Okay. Can you confirm if you have an old phpunit.xml file in the root? The one we have there for the binary to pick up and is checked into Git contains <phpunit bootstrap="tests/phpunit/bootstrap.integration.php">, which should behave exactly as before for the IDE use case. In fact, it should behave better than before, since previously the top-level phpunit.xml file (which we've only had since a short while) was configured for pure unit tests. Before last year, we didn't have a top-level file, which meant phpunit binary didn't work at all out of the box unless config params are set to where our suite.xml file was.

I'm guessing you have something in your IDE, that made it use tests/phpunit/suite.xml, which no longer exists now?

In T90875#7990627, @Daimona wrote:

One thing I noticed with the new system, in particular the new bootstrap, is that it loads every extension and skin in the respective directories. This is not ideal for me, because I have some duplicated extension folders (e.g. uncommitted WIPs), some extensions are not updated (because I don't use them), and in general, I fear that loading all extensions could slow everything down. I'm not sure how this can be fixed though, because I assume we don't want to load LocalSettings.php...

I don't think that is related to patches from this task, perhaps it was from the most recent patch on T240535: Clean up ExtensionRegistry autoloading mess, https://gerrit.wikimedia.org/r/c/773514 ?

In T90875#7990856, @Krinkle wrote:

Ack. Okay. Can you confirm if you have an old phpunit.xml file in the root?

No, I've always used the default one.

The one we have there for the binary to pick up and is checked into Git contains <phpunit bootstrap="tests/phpunit/bootstrap.integration.php">, which should behave exactly as before for the IDE use case.

It does, however we have:

require_once __DIR__ . "/bootstrap.php";

and

$directoryToJsonMap = [
	$GLOBALS['wgExtensionDirectory'] => 'extension*.json',
	$GLOBALS['wgStyleDirectory'] => 'skin*.json'
];

$extensionProcessor = new ExtensionProcessor();

foreach ( $directoryToJsonMap as $directory => $jsonFilePattern ) {
	foreach ( new GlobIterator( $directory . '/*/' . $jsonFilePattern ) as $iterator ) {
		$jsonPath = $iterator->getPathname();
		$extensionProcessor->extractInfoFromFile( $jsonPath );
	}
}

so it's reading extension.json for every extension in the extensions/ directory, which is what causes the issues I was seeing: for instance, if I have two copies of the same extension, I will get an error from ExtensionProcessor about that.

I'm guessing you have something in your IDE, that made it use tests/phpunit/suite.xml, which no longer exists now?

I used to use both tests/phpunit/phpunit.php + tests/phpunit/suite.xml and vendor/bin/phpunit + phpunit.xml.dist from PHPStorm, switching the configuration depending on whether I needed to run a unit or integration test, and both would work fine. Now I changed the config to always use vendor/bin/phpunit and phpunit.xml.dist, but I get the errors above.

In T90875#7991561, @kostajh wrote:

I don't think that is related to patches from this task, perhaps it was from the most recent patch on T240535: Clean up ExtensionRegistry autoloading mess, https://gerrit.wikimedia.org/r/c/773514 ?

Maybe? One thing I can say for sure is that I was able to run tests until a couple days ago, and yesterday I immediately upgraded MW to try the new system, and it started failing.

Change 804305 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] phpunit: Temporary revert entrypoint to phpunit.php

https://gerrit.wikimedia.org/r/804305

Change 803918 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/core@master] Revert "phpunit: Default to vendor/bin/phpunit, remove suites.xml"

https://gerrit.wikimedia.org/r/803918

Change 803918 merged by jenkins-bot:

[mediawiki/core@master] Revert "phpunit: Default to vendor/bin/phpunit, remove suites.xml"

https://gerrit.wikimedia.org/r/803918

Change 804526 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [DNM] phpunit: Default to vendor/bin/phpunit, remove suites.xml (take 2)

https://gerrit.wikimedia.org/r/804526

Change 804547 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] phpunit: Inherit Integration/Unit group from parent test case

https://gerrit.wikimedia.org/r/804547

Change 804305 abandoned by Krinkle:

[mediawiki/core@master] phpunit: Temporarily revert entrypoint to phpunit.php

Reason:

https://gerrit.wikimedia.org/r/804305

Change 804547 abandoned by Kosta Harlan:

[mediawiki/core@master] [WIP] phpunit: Inherit Integration/Unit group from parent test case

Reason:

https://gerrit.wikimedia.org/r/804547

Change 693566 abandoned by Daimona Eaytoy:

[mediawiki/core@master] [WIP] Delete the custom phpunit.php entry point

Reason:

https://gerrit.wikimedia.org/r/693566

Change 813198 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] phpunit.xml.dist: Align contents with suite.xml

https://gerrit.wikimedia.org/r/813198

Change 813199 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] phpunit: Move globals snapshot/reset code into TestSetup

https://gerrit.wikimedia.org/r/813199

Change 813200 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] phpunit: Restore unreachable code

https://gerrit.wikimedia.org/r/813200

Change 813199 abandoned by Kosta Harlan:

[mediawiki/core@master] phpunit: Move globals snapshot/reset code into TestSetup

Reason:

Meh, it's probably better for this code to stay in MW unit test case.

https://gerrit.wikimedia.org/r/813199

Change 813209 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] phpunit: Add Integration/Unit bootstrap files

https://gerrit.wikimedia.org/r/813209

Change 813198 merged by jenkins-bot:

[mediawiki/core@master] phpunit.xml.dist: Align contents with suite.xml

https://gerrit.wikimedia.org/r/813198

Change 813200 merged by jenkins-bot:

[mediawiki/core@master] phpunit: Restore unreachable code

https://gerrit.wikimedia.org/r/813200

Change 813271 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/ConfirmEdit@master] phpunit: Check for MW_PHPUNIT_TEST constant

https://gerrit.wikimedia.org/r/813271

Change 813271 merged by jenkins-bot:

[mediawiki/extensions/ConfirmEdit@master] phpunit: Check for MW_PHPUNIT_TEST constant

https://gerrit.wikimedia.org/r/813271

Change 813327 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] phpunit: Remove suite.xml, default to vendor/bin/phpunit

https://gerrit.wikimedia.org/r/813327

This is the main blocker to using new versions of paratest (e.g. beyond 5.0.4)

My sense from chipping away at this over the years is that this is overall a large project, that would require buy-in and support from WMF/WMDE teams and individuals across the Wikimedia developer community. And we'd need to be willing to revisit some assumptions of our test setup (e.g. that it would be supported to run PHPUnit in a production environment, or that one doesn't need to use the base test cases) and features (e.g. getting rid of the hook for discovering tests which requires bootstrapping MediaWiki, and using file system paths for discovery).

Because no team is owning our PHPUnit infrastructure, I am unsure how we could get resourcing, sponsorship and support to make this happen.

I think the last attempt at this got pretty close. The main issue I noticed is that we would need a way to specify a list of extensions whose tests should be executed, or it will just scan all of them, see T90875#7991774. Then we'd likely find other issues, but maybe they'd be easier to fix. In general I agree with Kosta that the PHPUnit infra is unmaintained, and with no group of people officially supporting it it's definitely harder to get things like this done.

Change 813209 abandoned by Kosta Harlan:

[mediawiki/core@master] phpunit: Add Integration/Unit bootstrap files

Reason:

https://gerrit.wikimedia.org/r/813209