Page MenuHomePhabricator

Remove old wmf/* deployment branches (recurring chore)
Closed, ResolvedPublic

Description

We don't have any real need for 1.23wmf* and 1.24wmf* deployment branches anymore. They should probably be changed to tags.

Chad, do you have a script for doing this? Timo seemed to think you had from before...


EDIT: Krinkle used the following script for individual repositories at https://github.com/Krinkle/dotfiles/tree/master/hosts/KrinkleMac/bin:

1#!/bin/bash -ue
2
3if [ -z "${1:-}" ]; then
4 # Missing parameters
5 echo
6 echo " Usage: $(basename $0) <version>"
7 echo
8 echo ' Options:'
9 echo ' <version> MediaWiki version, e.g. "1.25" or "1.27"'
10 echo
11 exit 1
12fi
13
14remote='origin'
15prefix=$1
16if (( $(echo "$prefix >= 1.27" | bc ) )); then
17 prefix="$prefix.0-"
18fi
19git remote update $remote
20git remote prune $remote
21# Examples: "wmf/1.26wmf21", "wmf/1.27.0-wmf.12".
22# Use a strict prefix with "wmf" mach to avoid disasted when passing "1.2"
23# (e.g. don't expand as "1.20 - 1.29" or "1.200")
24refs=`git for-each-ref --format='%(refname)' "refs/remotes/${remote}/wmf/${prefix}wmf*"`
25branches=()
26for ref in $refs; do
27 echo $ref
28 branch=$(echo $ref | cut -d '/' -f 4-6)
29 branches+=("$branch")
30done
31if [ -n "${branches:-}" ]; then
32 for branch in "${branches[@]}"; do
33 if git push $remote ":refs/heads/$branch" > /dev/null 2>&1; then
34 echo "... removed branch $branch"
35 else
36 echo "Could not remove $branch"
37 fi
38 done
39fi
40git remote prune $remote

1#!/bin/bash -ue
2
3if [ -z "${1:-}" ]; then
4 # Missing parameters
5 echo
6 echo " Usage: $(basename $0) <version>"
7 echo
8 echo ' Options:'
9 echo ' <version> MediaWiki version, e.g. "1.25"'
10 echo
11 exit 1
12fi
13
14remote='origin'
15version=$1
16git remote update $remote
17git remote prune $remote
18refs=`git for-each-ref --format='%(refname)' refs/remotes/${remote}/wmf/${version}wmf*`
19branches=()
20for ref in $refs; do
21 branch=$(echo $ref | cut -d '/' -f 4-6)
22 if git tag $branch $ref > /dev/null 2>&1; then
23 echo "... created tag $branch"
24 else
25 echo "Could not (re)create tag $branch"
26 fi
27 # Remove branches after publishing tags in case user is
28 # not authorised to publish forged tags in Gerrit.
29 branches+=("$branch")
30done
31echo "... publishing tags"
32git push $remote --tags
33if [ -n "${branches:-}" ]; then
34 for branch in "${branches[@]}"; do
35 if git push $remote ":refs/heads/$branch" > /dev/null 2>&1; then
36 echo "... removed branch $branch"
37 else
38 echo "Could not remove $branch"
39 fi
40 done
41fi
42git remote update $remote
43git remote prune $remote

Event Timeline

Reedy raised the priority of this task from to Needs Triage.
Reedy updated the task description. (Show Details)
Reedy added a project: MediaWiki-Core-Team.
Reedy changed Security from none to None.
Reedy added subscribers: Reedy, demon, Krinkle.

I created tags for wmf/1.23wmf* and wmf/1.24wmf* and deleted the branches.

Script:

#!/usr/bin/env bash
remote='origin'
version='1.24'
git remote update $remote
git remote prune $remote
refs=`git for-each-ref --format='%(refname)' refs/remotes/${remote}/wmf/${version}wmf*`
for ref in $refs; do
	branch=$(echo $ref | cut -d '/' -f 4-6)
	git tag $branch $ref > /dev/null 2>&1
	if [[ $? == 0 ]]; then
		echo "... tagged $branch"
	else
		echo "Could not (re)create $branch"
	fi
done
echo
echo "Ask a Gerrit admin to delete wmf/$version branches via https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/core,branches"
echo
echo "... publishing tags"
git push $remote --tags
git remote update $remote
git remote prune $remote
Krinkle renamed this task from Change old wmf deployment branches to tags to Convert old 1.23wmf* and 1.24wmf* deployment branches to tags.Nov 26 2014, 3:58 PM
Krinkle closed this task as Resolved.

Can this also be done for extensions as well?

I don't see any reason we couldn't. It would be running the same script essentially for all of our submodules

hashar added a subscriber: hashar.

That might break zuul-cloner which expects to checkout branches which is used for T1350: RFC: Extensions continuous integration among others. (link: https://www.mediawiki.org/wiki/RFC/Extensions_continuous_integration ).

I have filled T76088: Investigate whether zuul-cloner recognize tags just like branches to investigate whether zuul-cloner will be able to consider tags instead of branches.

@hashar At this point this is only happening to wmf branches. After a MediaWiki core wmf-branch is abandoned, we definitely wouldn't be backporting patches for an individual extension's wmf-branch as the relevant wmf-branch wouldn't be deployed anymore (and couldn't as you'd have to update the submodule pointer in the MediaWiki core branch that no longer exists).

For release branches this is different. Extension authors may maintain their extensions' release branch beyond MediaWiki core's end-of-life data for a release. But so far we haven't purged any release branches.

@Reedy @Chad As for wmf-branches in extensions, I'd like to remove them instead of converting to tags. Do we need them for anything? The submodule pointers from the wmf-branches in mediawiki-core refer to commits, not branches. Having said that, should we keep them even in MediaWiki core? It's not like we'd revert back an entire major release in production.

I have no real strong opinions about keeping them around.

Updated the script and published to P531. It now works for both MediaWiki core and extensions. And it also removes the branches afterward (instead of requiring manual action via Gerrit dashboard). Note that this requires Gerrit administrator permissions to run.

1#!/bin/bash -ue
2
3if [ -z "${1:-}" ]; then
4 # Missing parameters
5 echo
6 echo " Usage: $(basename $0) <version>"
7 echo
8 echo ' Options:'
9 echo ' <version> MediaWiki version, e.g. "1.25"'
10 echo
11 exit 1
12fi
13
14remote='origin'
15version=$1
16git remote update $remote
17git remote prune $remote
18refs=`git for-each-ref --format='%(refname)' refs/remotes/${remote}/wmf/${version}wmf*`
19branches=()
20for ref in $refs; do
21 branch=$(echo $ref | cut -d '/' -f 4-6)
22 if git tag $branch $ref > /dev/null 2>&1; then
23 echo "... created tag $branch"
24 else
25 echo "Could not (re)create tag $branch"
26 fi
27 # Remove branches after publishing tags in case user is
28 # not authorised to publish forged tags in Gerrit.
29 branches+=("$branch")
30done
31echo "... publishing tags"
32git push $remote --tags
33if [ -n "${branches:-}" ]; then
34 for branch in "${branches[@]}"; do
35 if git push $remote ":refs/heads/$branch" > /dev/null 2>&1; then
36 echo "... removed branch $branch"
37 else
38 echo "Could not remove $branch"
39 fi
40 done
41fi
42git remote update $remote
43git remote prune $remote

if there's no harm in keeping the tags around it sounds like a good idea to have them

I've ran the script for 1.25 on mediawiki/core and the following mediawiki extension repositories I have checked out:

CategoryTree
CentralNotice
CirrusSearch
Cite
Citoid
CodeReview
Collection
ContentTranslation
CvnDB
Echo
EventLogging
Flow
Gadgets
GlobalCssJs
GlobalUserPage
GuidedTour
ImageMetrics
Interwiki
Math
MobileFrontend
MultimediaViewer
ParserFunctions
Scribunto
Sentry
SyntaxHighlight_GeSHi
TemplateData
Thanks
Translate
UniversalLanguageSelector
VisualEditor
WikiEditor
WikimediaEvents
ZeroBanner
Krinkle renamed this task from Convert old 1.23wmf* and 1.24wmf* deployment branches to tags to Convert old wmf/* deployment branches to tags.Apr 17 2015, 5:53 PM
Krinkle triaged this task as Medium priority.
Krinkle edited projects, added Release-Engineering-Team; removed MediaWiki-Core-Team.
Krinkle renamed this task from Convert old wmf/* deployment branches to tags to Convert old wmf/* deployment branches to tags (recurring chore).Apr 29 2015, 8:21 PM
Krinkle removed Krinkle as the assignee of this task.

Maybe the make-wmf-branch could be taught how to convert obsolete branches to tags? This way the conversion will be part of the process and we could close this task.

(Re-opening, since there are still plenty of extensions with them, and it will happen again every release cycle)

Ran convert-wmf-branches 1.26 on repos mediawiki/core and mediawiki/extensions/{VisualEditor,NavigationTiming}.

Actually, I'd like to repurpose this as "Just kill old branches." The tags are useless and clutter shit up which is why I nuked all of them a few weeks ago.

Also we should prune old branches for extensions/skins/vendor too, I only do it for core right now.

One liner for all extensions and skins meta repos:

# Delete all remote branches matching wmf/ that aren't 1.28-based
git submodule foreach "git branch -r | grep -Eo 'wmf/.*' | grep -v '1.28' | xargs -I {} git push origin :{}"
# Delete all (remote) tags named wmf/* 
git submodule foreach "git tag | grep -Eo 'wmf/.*' | xargs -I {} git push origin :{} ||:"
# Delete the tags locally too, so we don't re-push them.
git submodule foreach "git tag | grep -Eo 'wmf/.*' | xargs -I {} git tag -d {}"

Just gotta swap 1.28 for whatever branch you're wanting to retain (the current one, obviously).

Just ran this for all skins and the vendor directory. Finishing up extensions now.

Let's please not reintroduce these tags. And get better (via the child task) at cleaning up the branches automatically.

I agree, all those old tags and branches aren't very useful after a while.

Done for all extensions, skins and vendor. No more extraneous tags. No more branches other than the currently active series, 1.28.0-wmf.N

Krinkle updated the task description. (Show Details)
This comment was removed by demon.

So I think the last thing to do here is to automate this a bit. I'm going to add it to scap clean

Krinkle renamed this task from Convert old wmf/* deployment branches to tags (recurring chore) to Remove old wmf/* deployment branches (recurring chore).Mar 11 2017, 12:44 AM
Krinkle awarded a token.
demon claimed this task.

This is done.