Page MenuHomePhabricator

mediawiki/vendor repo has lost all its REL branches
Closed, ResolvedPublic

Description

As pointed out by nik on #mediawiki on IRC, the mediawiki/vendor repo
appears to have lost all its REL branches, for example REL1_27.

They definitely used to exist, we even backported patches to them:
https://gerrit.wikimedia.org/r/#/q/status:merged+project:mediawiki/vendor+branch:REL1_27

But they are gone now:
https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/vendor,branches

It's not just a Gerrit interface issue, Git commands also don't know about
them:

mediawiki/vendor$ git branch --remote
  origin/HEAD -> origin/master
  origin/fundraising/REL1_27
  origin/master
  origin/wmf/1.31.0-wmf.15
  origin/wmf/1.31.0-wmf.16
  origin/wmf/1.31.0-wmf.17

This deletion seems to have been replicated to GitHub as well:
https://github.com/wikimedia/mediawiki-vendor/branches/all

Event Timeline

matmarex created this task.Jan 24 2018, 3:53 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 24 2018, 3:53 AM
Njk added a subscriber: Njk.Jan 24 2018, 3:03 PM
demon closed this task as Invalid.Jan 24 2018, 6:10 PM
demon added a subscriber: demon.

That was done on purpose. That repo is for WMF deploys, and we should build the branches using composer (like you would from master). Cf: T172927.

Njk added a comment.Jan 24 2018, 6:58 PM

Thanks for the info. I will update this documentation then, since it sounds like there's some outdated info here. https://www.mediawiki.org/wiki/Download_from_Git#Fetch_external_libraries

Legoktm reopened this task as Open.Jan 24 2018, 7:10 PM
Legoktm added a subscriber: Legoktm.

We still need the REL branches though. I guess someone just needs to recreate them?

No, we shouldn't need them. If CI is using them for testing, then CI is incorrect.

Fix CI instead of creating branches we don't want or need.

I feel like there's been some misunderstanding here. We made a point when adopting composer that people would still be able to install MediaWiki (plus bundled extensions) without needing to run composer themselves, as they could use mediawiki/vendor (second option on https://www.mediawiki.org/w/index.php?title=Download_from_Git&oldid=2697188#Fetch_external_libraries). We still need to have branches for that.

demon added a comment.Jan 25 2018, 3:01 PM

They can install it without running composer themselves--by using the tarball. But if you're cloning MediaWiki...well...that's why we have the "installing from Git" instructions. REL1_* branches are still Git... (also cf: gerrit 401659).

Unless you're proposing to put mediawiki/vendor in master too? Part of my motivation here has been trying to reduce the weird branch-specific build procedures...it complicates our tooling and documentation.

Krinkle added a subscriber: Krinkle.Feb 3 2018, 3:29 AM

@demon I think there is a misunderstanding somewhere, but I'm not sure exactly where.

As of late, we now have git submodules in mediawiki/core REL branches for skins and extensions. If I recall correctly, this was done so that tarballs are easier to create and verify, given their exact structure is mapped to a branch in Git. I suppose it also makes it easier for those who maintain MediaWiki installs from Git to perform the same upgrades as those who use tarballs, by simply updating the submodule pointers based on the commits you make to REL's gitmodules file for tarball releases.

However, I don't understand why mediawiki/vendor would be different. Surely we want the composer deps that go in the tarball to also be part of the snapshot we have in Git? For the same reasons of easy verifiability? Right now, it seems they are downloaded locally with composer on the release server for the tarball, but no reference is kept anywhere in Git. Is it mandatory now that when using REL branches from Git, one must install composer? Even if that were the official best practice, I'd still think we want to at least keep a reference around to the files we've published in Git so that at e.g. one can easily inspect which versions of upstream libs are being distributed in the tarball.

I know about T172927, but I interpreted that as meaning we want to create REL branches in vendor fresh, based on what we need for the bundled extensions (result of running composer merge and update, as you do for the tarball), and then commit that to vendor:REL1_*.

Legoktm claimed this task.Feb 3 2018, 3:38 AM

Chad and I discussed this at all-hands, I just haven't had a chance to post a summary of our plan going forward (and how I'm going to clean up the mess I unintentionally caused).

Legoktm closed this task as Resolved.Feb 7 2018, 10:14 PM

I restored the branches. The plan going forward for future releases is to continue creating branches, but we will filter down the list of installed libraries to just what is required by everything distributed by the tarball.