T248908: composer-merge-plugin support for composer 2.0 is our task for this

The upstream PR is still WIP

I don't believe there's any major features to be gained; in most cases, I don't think there's any good reason to rush upgrading, beyond the fact composer self-update etc by default does unbounded upgrades to 2.x

Yeah - the composer compatibility issue is the issue - I've managed to work around it for now in the docker image I'm trying to get going but we've had to fork an upstream repo to get there.

One of many problems of people using unbound/non specific depedancies. Stable releases change and then things break :)

Well yeah - in this case I suspect we would be better to fork composer-merge plugin & unfork the other but we'll see what happens

You’ve got upstream commit access...

Wouldn’t it be easier to just pin (or download 1x using https://getcomposer.org/composer-1.phar rather than the generic unspecific urls) composer there? I’m presuming no one has a workflow that means they *must* have composer 2.x yet

Forking the merge plugin when it doesn’t have a complete fix seems excessive

Well at the moment we have forked an open source library that we use heavily in order to lock composer to 1.0 for the merge-plugin - but it feels like the tail is wagging the dog. If the work on the composer merge plugin is kinda moving then it makes sense to just wait a bit

@AMooney and @MNadrofsky, here is more support for your teams working on T248908: composer-merge-plugin support for composer 2.0

@AMooney & @MNadrofsky also we would send you virtual hugs....

Change 664514 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Update composer merge plugin

https://gerrit.wikimedia.org/r/664514

Composer merge plugin is fixed upstream now. Lets

1. get onto version 2
2. remove our buildkit version 1 hack
3. switch CI & our dev installs to use the upstream buildkit & stop using our one

Would also close out https://phabricator.wikimedia.org/T126266

I moved this back into triage as getting this done will remove an ongoing pain point and the work involved now is probably less than the ongoing pain

Change 669977 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Upgrade wikimedia/composer-merge-plugin to version 2

https://gerrit.wikimedia.org/r/669977

Change 669993 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm/civicrm-buildkit@master] Switch to composer 2

https://gerrit.wikimedia.org/r/669993

It seems that updating composer-merge-plugin without running a free range composer update is tricky - just recording our current install set

In T267881#6894388, @Eileenmcnaughton wrote:

It seems that updating composer-merge-plugin without running a free range composer update is tricky

Using composer require --update-no-dev wikimedia/composer-merge-plugin 2.0.1 should do that for you...

thanks @Reedy - I'll try that!

Wow - that is taking forever to run .... waiting waiting

Hmm - nope doing that it removed a bunch of packages added in by composer merge plugin (as well as some dev packages which are legit removals)

docker@civicrm:/srv/civi-sites/wmff$ composer require --update-no-dev wikimedia/composer-merge-plugin ^2

./composer.json has been updated
Loading composer repositories with package information Updating dependencies
Package operations: 0 installs, 1 update, 52 removals

• Removing zordius/lightncandy (v0.23)
• Removing willdurand/geocoder (dev-master)
• Removing wikimedia/testing-access-wrapper (1.0.0)
• Removing wikimedia/omnimail-silverpop (dev-master)
• Removing wikimedia/civicrm-data-table-provider (4.0.0)
• Removing theseer/tokenizer (1.1.3)
• Removing seld/jsonlint (1.7.0)
• Removing sebastian/version (2.0.1)
• Removing sebastian/resource-operations (2.0.1)
• Removing sebastian/recursion-context (3.0.0)
• Removing sebastian/object-reflector (1.1.1)
• Removing sebastian/object-enumerator (3.0.3)
• Removing sebastian/global-state (2.0.0)
• Removing sebastian/exporter (3.1.2)
• Removing sebastian/environment (4.2.3)
• Removing sebastian/diff (3.0.2)
• Removing sebastian/comparator (3.0.2)
• Removing sebastian/code-unit-reverse-lookup (1.0.1)
• Removing phpunit/phpunit (7.5.20)
• Removing phpunit/php-token-stream (3.1.1)
• Removing phpunit/php-timer (2.1.2)
• Removing phpunit/php-text-template (1.2.1)
• Removing phpunit/php-file-iterator (2.0.2)
• Removing phpunit/php-code-coverage (6.1.4)
• Removing phpspec/prophecy (1.8.0)
• Removing phpdocumentor/type-resolver (0.4.0)
• Removing phpdocumentor/reflection-docblock (3.3.2)
• Removing phpdocumentor/reflection-common (1.0.1)
• Removing php-http/promise (v1.0.0)
• Removing php-http/message-factory (v1.0.2)
• Removing php-http/message (1.6.0)
• Removing php-http/httplug (v1.1.0)
• Removing php-http/guzzle6-adapter (v1.1.1)
• Removing php-http/discovery (1.4.0)
• Removing phar-io/version (2.0.1)
• Removing phar-io/manifest (1.0.3)
• Removing omnimail/omnimail (dev-master)
• Removing myclabs/deep-copy (1.7.0)
• Removing mrmarkfrench/silverpop-php-connector (1.5.0)
• Removing league/csv (9.3.0)
• Removing jakub-onderka/php-parallel-lint (v0.9.2)
• Removing jakub-onderka/php-console-highlighter (v0.3.2)
• Removing jakub-onderka/php-console-color (0.1)
• Removing geocoder-php/nominatim-provider (5.1.0)
• Removing geocoder-php/mapquest-provider (dev-master)
• Removing geocoder-php/here-provider (dev-master)
• Removing geocoder-php/google-maps-provider (4.4.0)
• Removing geocoder-php/free-geoip-provider (4.1.0)
• Removing geocoder-php/common-http (4.2.0)
• Removing geo6/geocoder-php-addok-provider (dev-master)
• Removing doctrine/instantiator (1.3.1)
• Removing clue/stream-filter (v1.4.1)
• Updating wikimedia/composer-merge-plugin (v1.4.1 => v2.0.1): Loading from cache

Are they all dev dependancies? Numerous of them look to be...

--no-update                     Disables the automatic update of the dependencies.
--no-scripts                    Skips the execution of all scripts defined in composer.json file.
--update-no-dev                 Run the dependency update with the --no-dev option.
--update-with-dependencies      Allows inherited dependencies to be updated, except those that are root requirements.

So maybe composer require --no-update wikimedia/composer-merge-plugin ^2 might be what you want

A number of them are but not all - so specifically

Removing league/csv (9.3.0)

is one that should not being going anywhere

Also some are legitimately using dev-master such as willdurand/geocoder (dev-master)

This sequence DOES work (although it updates extras)

• Manually update composer.json to have version ^2 for composer-merge-plugin
• run composer update with no switches
• run composer install --no-dev

I'll try your command now though

dev in this case should means things that are listed in a require-dev somewhere in the heirarchy; following dev-master as the version shouldn't be an issue.

Depending on where it is in the tree, you might want to just explicity add league/csv to the parent composer.json, rebuild, and then try updating...

It was one thing we noticed during the CR of the plugin update, a few behaviours didn't necessarily always run as expected. Worse case, should be running update twice to bring things back again.

Of course, when various dependancies are not explicitly pinned, things will change... So running composer update (with or without --no-dev) first, before changing the plugin etc might help make it all cleaner to review

Yep - I'm just looking to see what composer update will change - it would be nice just to get it all updated in general & if that makes this easier then it might be a good chance to do it

I'm going backwards - nothing seems to work now. I will take a break

OK - I found an unrelated issue was messing with me - trying again

Dang - now CI is failing

14:25:30 Running composer update to apply merge settings
14:25:30 Cannot create cache directory /cache/composer/repo/https---repo.packagist.org/, or directory is not writable. Proceeding without cache
14:25:30 Cannot create cache directory /cache/composer/files/, or directory is not writable. Proceeding without cache
14:25:30 Loading composer repositories with package information
14:25:30
14:25:30
14:25:30 [RuntimeException]
14:25:30 /cache/composer/vcs does not exist and could not be created.
14:25:30
14:25:30

So my thinking on this is that we need to either

1. create the cache dir when installing or
2. make it a volume - there seems to be stuff on the web indicating this would speed it up as the cache would be shared between runs. This would not necessarily cause issues as the cache should be simplfy downloads

Side note - this kept popping up https://theiconic.tech/how-speeding-up-composer-install-led-to-reducing-php-docker-image-build-time-by-up-to-5-times-12c024f06924

Change 669977 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Upgrade wikimedia/composer-merge-plugin to version 2

https://gerrit.wikimedia.org/r/669977

Change 669993 merged by Eileen:
[wikimedia/fundraising/crm/civicrm-buildkit@master] Switch to composer 2

https://gerrit.wikimedia.org/r/669993

Change 664514 abandoned by Eileen:
[wikimedia/fundraising/crm@master] Update composer merge plugin

Reason:

https://gerrit.wikimedia.org/r/664514