Page MenuHomePhabricator

mediawiki/vendor repo has lost all its REL branches
Closed, ResolvedPublic

Description

As pointed out by nik on #mediawiki on IRC, the mediawiki/vendor repo
appears to have lost all its REL branches, for example REL1_27.

They definitely used to exist, we even backported patches to them:
https://gerrit.wikimedia.org/r/#/q/status:merged+project:mediawiki/vendor+branch:REL1_27

But they are gone now:
https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/vendor,branches

It's not just a Gerrit interface issue, Git commands also don't know about
them:

mediawiki/vendor$ git branch --remote
  origin/HEAD -> origin/master
  origin/fundraising/REL1_27
  origin/master
  origin/wmf/1.31.0-wmf.15
  origin/wmf/1.31.0-wmf.16
  origin/wmf/1.31.0-wmf.17

This deletion seems to have been replicated to GitHub as well:
https://github.com/wikimedia/mediawiki-vendor/branches/all

Event Timeline

demon subscribed.

That was done on purpose. That repo is for WMF deploys, and we should build the branches using composer (like you would from master). Cf: T172927.

Thanks for the info. I will update this documentation then, since it sounds like there's some outdated info here. https://www.mediawiki.org/wiki/Download_from_Git#Fetch_external_libraries

Legoktm subscribed.

We still need the REL branches though. I guess someone just needs to recreate them?

No, we shouldn't need them. If CI is using them for testing, then CI is incorrect.

Fix CI instead of creating branches we don't want or need.

I feel like there's been some misunderstanding here. We made a point when adopting composer that people would still be able to install MediaWiki (plus bundled extensions) without needing to run composer themselves, as they could use mediawiki/vendor (second option on https://www.mediawiki.org/w/index.php?title=Download_from_Git&oldid=2697188#Fetch_external_libraries). We still need to have branches for that.

They can install it without running composer themselves--by using the tarball. But if you're cloning MediaWiki...well...that's why we have the "installing from Git" instructions. REL1_* branches are still Git... (also cf: gerrit 401659).

Unless you're proposing to put mediawiki/vendor in master too? Part of my motivation here has been trying to reduce the weird branch-specific build procedures...it complicates our tooling and documentation.

@demon I think there is a misunderstanding somewhere, but I'm not sure exactly where.

As of late, we now have git submodules in mediawiki/core REL branches for skins and extensions. If I recall correctly, this was done so that tarballs are easier to create and verify, given their exact structure is mapped to a branch in Git. I suppose it also makes it easier for those who maintain MediaWiki installs from Git to perform the same upgrades as those who use tarballs, by simply updating the submodule pointers based on the commits you make to REL's gitmodules file for tarball releases.

However, I don't understand why mediawiki/vendor would be different. Surely we want the composer deps that go in the tarball to also be part of the snapshot we have in Git? For the same reasons of easy verifiability? Right now, it seems they are downloaded locally with composer on the release server for the tarball, but no reference is kept anywhere in Git. Is it mandatory now that when using REL branches from Git, one must install composer? Even if that were the official best practice, I'd still think we want to at least keep a reference around to the files we've published in Git so that at e.g. one can easily inspect which versions of upstream libs are being distributed in the tarball.

I know about T172927, but I interpreted that as meaning we want to create REL branches in vendor fresh, based on what we need for the bundled extensions (result of running composer merge and update, as you do for the tarball), and then commit that to vendor:REL1_*.

Chad and I discussed this at all-hands, I just haven't had a chance to post a summary of our plan going forward (and how I'm going to clean up the mess I unintentionally caused).

I restored the branches. The plan going forward for future releases is to continue creating branches, but we will filter down the list of installed libraries to just what is required by everything distributed by the tarball.