Page MenuHomePhabricator

Unable to update translatewiki.net: romaricdrigon/metayaml 1.1 requires symfony/yaml ~2.3|~3.0
Closed, ResolvedPublic

Description

This is a conflict between a dependency on Translate and MediaWiki core.

Event Timeline

Nikerabbit triaged this task as Unbreak Now! priority.Oct 14 2019, 7:26 AM
Nikerabbit added subscribers: Reedy, Jdforrester-WMF.

Caused by rMWcd6f99e97f71: Upgrade symfony/yaml from 3.4.31 to 4.3.4. I see a few options:

  • Ask upstream to allow 4.x
  • Relax MediaWiki requirements to allow 3.x|4.x (as far as I know nothing depends on 4.x)
  • Fork upstream to allow 4.x (meh)
  • Replace metayaml with something else (not a good use of time)
  • Stop using metayaml (don't want to drive without seatbelts)

Master has a fix for that, so it could mostly just be asking them to release the fix - https://github.com/romaricdrigon/MetaYaml/commit/1481b99679b29cbb50c88f3e10e4951ac64b8ad3

https://github.com/romaricdrigon/MetaYaml/compare/1.1...master

Certainly relaxing the requirement in MW (to allow 3 or 4) seems reasonable in the meantime

Okay, I'll ask them about doing a release.

Currently carrying a local patch to override composer.json.

The local patch just had a merge conflict.

Change 545780 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/core@master] Allow symfony/yaml 3.x

https://gerrit.wikimedia.org/r/545780

Change 545780 merged by jenkins-bot:
[mediawiki/core@master] composer: Allow symfony/yaml ~3.4

https://gerrit.wikimedia.org/r/545780

Still silent in upstream. Should we close this task now?

abi_ lowered the priority of this task from Unbreak Now! to High.Nov 18 2019, 10:01 AM
abi_ subscribed.

Reducing the priority to High since we've made a local patch.

Change 570621 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/core@master] composer: Re-add symfony/yaml 3.4

https://gerrit.wikimedia.org/r/570621

Change 570621 merged by jenkins-bot:
[mediawiki/core@master] Relax symfony/yaml to allow ~3.4|~4.5|~5.0

https://gerrit.wikimedia.org/r/570621

Upstream seems a bit dead...

Change 638780 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/core@master] Update symfony/yaml supported versions

https://gerrit.wikimedia.org/r/638780

Change 638780 merged by jenkins-bot:
[mediawiki/core@master] Update symfony/yaml supported versions

https://gerrit.wikimedia.org/r/638780

Still used today in tests/phunit/integration/RequestLoggerTest.php. No reason it has to be this particular yaml library or version. Alternatively, EventBus could provide a way to validate events against a named schema from a test suite so we don't have to do it ourselves.

Still used today in tests/phunit/integration/RequestLoggerTest.php. No reason it has to be this particular yaml library or version. Alternatively, EventBus could provide a way to validate events against a named schema from a test suite so we don't have to do it ourselves.

Is this issue why the requirement is needed in core's composer.json resolved?

Or can we just add it to MediaWiki-Vendor instead?

Change 654329 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/vendor@master] Add symfony/yaml 5.1.8

https://gerrit.wikimedia.org/r/654329

Change 654326 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/core@master] WIP: Remove symfony/yaml from require-dev

https://gerrit.wikimedia.org/r/654326

@Nikerabbit I presume TranslateWiki doesn't actually use the mediawiki-vendor repo, right?

Looks like this works fine. I think at one point, MediaWiki-extensions-OpenStackManager used to have symfony/yaml in MediaWiki-Vendor

I guess the problem is that we don't run composer update/install in CI for WMF deployed stuff, so symfony/yaml doesn't get installed at CI time. Putting it in vendor fixes that...

But does that break CI for Translate?

Change 653999 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/Translate@master] DMN: Test with symfony/yaml removed from MW Core require-dev

https://gerrit.wikimedia.org/r/653999

Change 653997 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/CirrusSearch@master] DMN: Test with symfony/yaml removed from MW Core require-dev

https://gerrit.wikimedia.org/r/653997

Looks like this works fine. I think at one point, MediaWiki-extensions-OpenStackManager used to have symfony/yaml in MediaWiki-Vendor

I guess the problem is that we don't run composer update/install in CI for WMF deployed stuff, so symfony/yaml doesn't get installed at CI time. Putting it in vendor fixes that...

But does that break CI for Translate?

It should yes, unless we are failing to validate dependency constraints there, which would be a bug, since 1) Translate and Cirrus are both WMF production extensions, and 2) The only constraint validation logic we have is that of Composer, which doesn't currently have a way to enforce one without the other afaik. (If it had a way to do just one of them, we wouldn't need the hacky require-dev extraction we have in Quibble today, and thus we could do this all much simpler by letting it be declared in the composer.json/require-dev of Cirrus).

Looks like this works fine. I think at one point, MediaWiki-extensions-OpenStackManager used to have symfony/yaml in MediaWiki-Vendor

I guess the problem is that we don't run composer update/install in CI for WMF deployed stuff, so symfony/yaml doesn't get installed at CI time. Putting it in vendor fixes that...

But does that break CI for Translate?

It should yes, unless we are failing to validate dependency constraints there, which would be a bug, since 1) Translate and Cirrus are both WMF production extensions, and 2) The only constraint validation logic we have is that of Composer, which doesn't currently have a way to enforce one without the other afaik. (If it had a way to do just one of them, we wouldn't need the hacky require-dev extraction we have in Quibble today, and thus we could do this all much simpler by letting it be declared in the composer.json/require-dev of Cirrus).

The library in question in Translate is only a suggest. And AFAIK we definitely don't tell composer to install suggests, so it doesn't fail in our CI

Change 654329 abandoned by Reedy:

[mediawiki/vendor@master] Add symfony/yaml 5.2.2

Reason:

https://gerrit.wikimedia.org/r/654329

Change 654326 abandoned by Reedy:

[mediawiki/core@master] Remove symfony/yaml from require-dev

Reason:

Promoted to require in master

https://gerrit.wikimedia.org/r/654326

Change 653999 abandoned by Reedy:

[mediawiki/extensions/Translate@master] DMN: Test with symfony/yaml removed from MW Core require-dev

Reason:

https://gerrit.wikimedia.org/r/653999

Just to note MW core now has symfony/yaml in require these days..

Nikerabbit claimed this task.

Upstream still silent. I guess at some point we may need to switch over to another validation library... preferably something that's already used in MediaWiki or other extension.