Page MenuHomePhabricator

Add support for git branches to ExtensionDistributor
Closed, ResolvedPublic

Description

Right now there is general git support (T37574), but not for branches yet.

The only place Git is supported is as replacement for 'trunk'. I'm not sure that would even work, actually, since then

  • extensions that are only in SVN would no longer be downloadable
  • branches for Git-only extensions not visible
  • if an extension is in both, the branches from the outdated dir in SVN would be shown together with an option for 'master' form Git

.. which is probably why it is disabled by default and we're not using it on mediawiki.org yet.

Instead we should use the local branches of the extensions. If we want to we could have a configuration option that only shows branch names that are in a whitelisted array.

Also, we likely need to get rid of the hardcoded path requirement that everything has to follow [mediawiki]/branches/$version/$extensionName.


Version: unspecified
Severity: major

Details

Reference
bz37946

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 12:27 AM
bzimport set Reference to bz37946.

Here's an idea, let me know what you think:

array(

'Cite' => array(
    'dir' => '/var/extdist-workingcopies/gerrit/mediawiki/extensions/Cite',
    'branches' => true,
),
'CreateAPage' array(
    'branchDirs' => array(
        'trunk' => '/var/mw-snapshot/wikimedia-svn/trunk/extensions/Cite',
        'REL1_18' => '/var/mw-snapshot/wikimedia-svn/branches/REL1_18/extensions/Cite',
),

);

  • Adapters each have their own validator. Git supports any of the three properties, but requires at least one of 'dir' or 'branchDir' to be set. SVN only supports only 'dir' (as short cut to 'branchDir' => { trunk: dir }) and 'branchDir'.

Where 'branches' can be either:

  • - true (default): all branches it can automatically "find"
  • - array: whitelist of branches from the auto-find

And 'branchDirs' for using different directories for branches (mostly svn, though a git extension could use this too, e.g. branches from a fork in another repository clone directory).

Since this array would be huge and unlikely to be maintained manually, perhaps do this from a Hook (that would also save memory when viewing unrelated pages). For the Wikimedia install we'd have the working copy organized the same as now (with the addition of git):

/var/mw-snapshot/

  • wikimedia-svn/
    • trunk/
      • extensions/*
    • branches/
      • RELx_yy/
        • extensions/*
  • wikimedia-gerrit/
    • mediawiki/extensions/*

The hook would traverse these directories and format return array (which the hook caller should at least cache within the request). This would basically move the logic we currently have inside ExtensionDistributor to wmf-config.

Aside from that, something that the Git adapter still needs (either way) is to switch branches before making the tarball (and properly git-clean/git-reset as well, since switching branches has the tendency of leaving loose ends in Git).

Some of the things one will encounter in implementing something like that I recently faced and fixed for mediawiki-core:

(except that that script updates and tarballs from cron for all branches, whereas this should do that on-demand).

This should have a dependency on actually doing the branching on the git side, then this be for ED implementation

(In reply to comment #3)

This should have a dependency on actually doing the branching on the git side,
then this be for ED implementation

Added bug 38012 as a dependency.

All extensions bundled with 1.20.0 have a REL1_20 branch. And various others as well. Lets get this set up.

  • Bug 42365 has been marked as a duplicate of this bug. ***

Raising priority and tagging as regression. This is effectively a regression from the git migration until equaled.

Gerrit change 37478 was merged, so marking this fixed.

Will be deployed as part of the 1.21wmf8 cycle (Jan 16th).

Discussion ongoing on wikitech-l regarding branch/tagging strategies.