Related workaround: https://gerrit.wikimedia.org/r/#/c/mediawiki/vendor/+/503068/

Indeed extensions development dependencies are not fetched by the composer-merge-plugin and that is unfortunately on purpose :/ The composer.json of each extensions would most definitely have some version mismatches on various dev dependencies and such a merge would never pass.

I remember we had some discussions on that front with the hamcrest library though it benefited any extensions and core. So that ended up in mediawiki/core require-dev. Reference:

• Add hamcrest to require-dev https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/335814/
• Replacement of PHPUnit obsolete methods with hamcrest matchers https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/333961/

require-dev in extensions was at first just for composer install && composer test, there is no way though in composer.json to disambiguate dependencies that should be merged from some that should not. We are really hitting the limit of our systems and need a new definition? Maybe an extra field for the composer-merge-plugin which will let it merge some specific dev dependencies but none of the others.

Maybe Platform Engineering can assist on that front.

pinging @bd808 for input

Added TechCom for visibility. This isn't a system Core Platform runs, maybe something for SRE?

In T220723#5120325, @daniel wrote:

pinging @bd808 for input

@hashar is the local expert on the CI integration related to composer usage. He was the awesome human who came up with the initial implementation of mediawiki/vendor.git and how to use it in CI and production to satisfy the security concerns of both environments.

In T220723#5116729, @hashar wrote:

Maybe an extra field for the composer-merge-plugin which will let it merge some specific dev dependencies but none of the others.

This is possible, but probably also fragile. I'm not sure that there is a really good automated solution for what are essentially integration tests across components with uncoordinated "require-dev" dependency declarations. In order to perform the integration, someone must define the runtime of the system in a way that resolves the conflicts that may arise from combining things that were not implemented as a fixed suite. Composer has built-in capabilities for this in its internal DependencyResolver component. That capability has limitations however in that it can only find resolution if all declarations for a given library have some area of intersection. If asked to resolve a conflict between a request for "foo:>1.0,<2.0" and "foo:>=2.0" it will fail and require a human to figure out how to upgrade/downgrade the specifications for foo to allow for a single version satisfying both constraints.

One way to solve this would be just to implement what @EBernhardson has asked for: full use of require-dev across all Composer components involved in the test. @hashar is certainly correct that this will lead to conflicts in certain combinations. These would however become blockers to a clean test and thus things that humans would be able to notice and resolve. We already have semi-automated processes for things like version bump of mediawiki-codesniffer across a large collection of PHP projects. Expanding that to include other common development dependencies such as phpunit seems possible.

An additional complexity to keep in mind will be extensions that depend on other extensions. The initial workaround we deployed for this ran into issues, it worked for CirrusSearch and mw-core, but extensions that depended on CirrusSearch were failing in CI.

Change 513357 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/vendor@master] Add CirrusSearch dev dependency to require-dev

https://gerrit.wikimedia.org/r/513357

Change 513358 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/core@master] Add CirrusSearch dev dependency to require-dev

https://gerrit.wikimedia.org/r/513358

This was previously discussed at T199440: Install cache/integration-tests with Wikibase CI tests FWIW.

Change 513371 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/extensions/CirrusSearch@master] Validate request logging with json schema

https://gerrit.wikimedia.org/r/513371

Change 513358 merged by jenkins-bot:
[mediawiki/core@master] Add CirrusSearch dev dependency to require-dev

https://gerrit.wikimedia.org/r/513358

Change 513357 abandoned by EBernhardson:
Add CirrusSearch dev dependency to require-dev

Reason:
unnecessary.

https://gerrit.wikimedia.org/r/513357

This task should be retitled now that HHVM is gone. Leaving it alone for now since I'm not sure whether or not composer-merge-plugin is providing some of what's requested here.

Change #1296618 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/core@master] Add fig/log-test to require-dev

https://gerrit.wikimedia.org/r/1296618

@Krinkle: The new title is actively wrong. The entire point of vendor is that CI jobs don't install anything, they use it as-is, pristinely.

I agree with you that installing composer deps is contrary to what "vendor" jobs are for, but please note that this is exactly what this task is asking for.

I'll make two notes:

1. The Phab task title is the same as before. I've only removed the outdated "hhvm" part from the job name, and clarified based on the task description that this is specifically about require-dev's specified in extension repos, and specifically about the Quibble job that runs PHPUnit.

2. Our "vendor" jobs already install extra deps from Composer. What's important, however, is that today this is strictly limited to adding on top of vendor (not changing or removing anything from vendor) and specifically only dev-dependencies. See Quibble source.

This is actually how we install phpunit, testing-access-wrapper, codesniffer, psysh, etc. in Vendor jobs today. I implemented that a decade ago as part of T112895 (pre-Quibble) to control the PHPUnit version in composer.json. (It's hard to imagine now, but we used to have our own packages for npm and phpunit globally installed on the Jenkins host; thus no consistency between local and CI, and these were hard to discover and maintain).

What's awkward today is that, if an extension needs an additional dev dependency, they have to add it to core's composer.json even if it isn't used by core. This is, for example, why we carry hamcrest and giorgiosironi/eris in MediaWiki core require-dev, despite only being used by Wikibase.

The benefit of course is that this means there's naturally no conflicts, and it ensures a consistent composer footprint across MW core and WMF extensions. For non-WMF extensions this would be unacceptable (for both parties) but those repos don't need to use vendor, so there's no problem there.

If we all agree on this status quo trade-off and don't see a need for change, then perhaps all we need to do is to document that the preferred way to use a new Composer dev dependency in an WMF-deployed extension, is to add it to MediaWiki core's composer.json (just like Wikibase did for Hamcrest and Eris), and accept that this adds overhead to CI jobs for MW core and extension repos, and for all people locally developing MediaWiki core (i.e. treat it as expensive, keep the count low.)

Change #1296618 merged by jenkins-bot:

[mediawiki/core@master] Add fig/log-test to require-dev

https://gerrit.wikimedia.org/r/1296618