Page MenuHomePhabricator

Provide a way to install Composer dependencies after installing an extension, without updating all unrelated libraries
Open, Needs TriagePublic

Description

When installing/updating an extension which has Composer dependencies, there are three approaches:

  1. run composer update in the main MediaWiki directory. This will cause composer to collect all dependencies (including the dependencies of the changed/new extension) via composer-merge-plugin, and update them. The result is always correct but will update every library to the newest version allowed by composer.json, not just the ones for the newly installed extension. Besides taking longer, more code changes means a higher probability of something breaking, thus more effort needed to test that the site works correctly.
  2. run composer install in the extension directory. This usually works (as long as the extension is able to use the autoloader from its vendor subdirectory, e.g. by setting the load_composer_autoloader flag in extension.json), and does not do unnecessary updates. It can fail in one edge case though: when the extension uses a library that is also used by another extension or MediaWiki core, Composer is not able to merge the version requirements and calculate the correct version (or throw an error if they conflict). Thus, there will be two different versions of the library, and callers might end up with a library version they did not expect.
  3. run composer update --with-dependencies library/one library/two ... to manually limit updates to the libraries needed by the extension. This works but is fairly cumbersome.

It would be nice to have a simpler way to install/update the affected libraries only.

Upstream bug: #6601

Event Timeline

Tgr created this task.Aug 11 2017, 8:41 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 11 2017, 8:41 PM
Tgr added a comment.Aug 11 2017, 8:42 PM

This is an upstream issue that we probably can't do much about. I mostly just created the task to have a description of the issue that can be referenced from on-wiki documentation.

Tgr updated the task description. (Show Details)Aug 11 2017, 8:42 PM
Samwilson added a subscriber: Samwilson.EditedSep 18 2017, 2:25 AM

This is one of the problems that composer.lock is supposed to fix, because then all dependencies are locked to a specific commit and cannot be changed. The other part of that idea is that one should never run composer update in production, but only ever composer install.

As for not wanting to update all dependencies at once, I think the current idea is that every composer file (core and extensions) should be specifying exact patch-level version numbers, and so (short of maintainers being naughty and changing a tagged version) there shouldn't be any unexpected upgrades.

I don't think there's any need to have any system of only updating some dependencies (other than, as you mention, the existing functionality in Composer). Rather, we should work towards a system in which a wiki can be deployed in a controlled way with its own composer.lock file (see also T166956, esp. the comment there "we don't trust upstream to follow semver", which I think is pertinent to this issue).

ahmad added a subscriber: ahmad.Apr 15 2018, 2:29 PM
run `composer install` in the extension directory.

This would create multiple vendor directories which is an anti-pattern. It would mean that if there are conflicting dependencies (or dependencies in multiple versions) then those discrepancies will not be resolved and will have to be resolved manually. :(

I suppose this is why we have MediaWiki-Vendor (see T178137)

Tgr added a comment.May 10 2018, 5:17 PM

It would mean that if there are conflicting dependencies (or dependencies in multiple versions) then those discrepancies will not be resolved and will have to be resolved manually. :(

And causes other problems like T186181. Nevertheless this is what we do currently in MediaWiki-Vagrant at least. Not sure which is the lesser evil.

With the vendor repo this is not really an issue since it is hand-curated (no on will run unrestricted composer update on it).

Seb35 added a subscriber: Seb35.Jan 11 2019, 5:49 PM

I read this bug and I am interested if it could be solved in some way. I’ve not tested yet, but if I understand correctly @dbarratt on issue #6601, there are some Composer commands which, when executed in a specific order, would solve this issue, isn’t it?

This next idea is far beyond this task, but reading this bug and having in mind other issues about MediaWiki + Composer + composer-merge-plugin + MediaWiki vs Composer autoloader + activation of MW extensions installed through Composer in a farm + numerous reactions from Composer team saying we are a very specific use case (which is probably true), I was wondering if MediaWiki should not create a dedicated binary about "installing vendor and possibly extensions/skins" and using Composer library and Packagist, but not the Composer CLI.

Tgr added a comment.Jan 11 2019, 6:03 PM

I read this bug and I am interested if it could be solved in some way. I’ve not tested yet, but if I understand correctly @dbarratt on issue #6601, there are some Composer commands which, when executed in a specific order, would solve this issue, isn’t it?

Well the idea there is to basically walk the dependency graph manually, as directed by error messages. It's possible of course, but not a particularly good user experience.

I was wondering if MediaWiki should not create a dedicated binary about "installing vendor and possibly extensions/skins" and using Composer library and Packagist, but not the Composer CLI.

I don't think anyone disagrees that's the ideal solution (there was more discussion on it in one of the extension management RfCs). Question is if/when it will actually happen.

@Seb35 You might look at T166956 for the "MW extensions installed through Composer" side of it. I haven't tried doing so in a farm, I ran into a bunch of issues (mostly to do with the file paths) in the installer which was a black hole of despair.

Kghbln added a subscriber: Kghbln.
Krinkle updated the task description. (Show Details)Tue, Mar 26, 8:31 PM
Krinkle added a subscriber: Krinkle.

@Seb35 You might look at T166956 for the "MW extensions installed through Composer" side of it. I haven't tried doing so in a farm, I ran into a bunch of issues (mostly to do with the file paths) in the installer which was a black hole of despair.

I understand the benefits of not having to edit a composer.local.json file. But it is unclear to why Composer's behaviour would be different between option 1 in this task description (composer-install in mediawiki dir), and a composer command in a Composer-managed project.

Do they behave differently? Is that a bug in Composer? If not, is there another way to solve it other than for Composer to offer a feature it currently doesn't have?

@Krinkle If I understand this correctly, if you have something like this in your composer.local.json:

{
	"extra": {
		"merge-plugin": {
			"include": [
				"extensions/GraphQL/composer.json"
			]
		}
	}
}

and you were to update the GraphQL extension, you would have no idea which packages might need to be updated. You have no idea what is in extensions/GraphQL/composer.json unless you manually go look at it, but even then you would need to do a diff to see if anything need to be removed.

The only work-around would be to do a composer update which would update literally everything.

The ideal solution would be to install the extension itself with Composer like so:

composer require mediawiki/extension-graphql

(or something like that)
That way you could then do:

composer update mediawiki/extension-graphql

and it would update only the extension and the dependencies of the extension.