Page MenuHomePhabricator

Rename Gerrit wikispeech_mockup repo to wikispeech-server
Closed, ResolvedPublic1 Estimated Story Points

Description

We will actually create a new, merge in the old and then delete the old. But we are waiting for a couple of blubber patches in gerrit that should merge to the old repo first.

Event Timeline

kalle added a project: User-kalle.
kalle updated the task description. (Show Details)
kalle set the point value for this task to 2.
kalle moved this task from 🥴 Backlog to 🤠 This week on the User-kalle board.

New repo created. When we merge in current patches in gerrit to the old, we'll merige it in to the new repo and then request the old to be removed.

This should also include updating the blubber build names and docker compose dependencies.

Lokal_Profil changed the point value for this task from 2 to 8.Jun 11 2020, 8:35 AM

This is blocked by similar problems as described in T200785

IRC log with relengs:

17:59 < kalle> When merging in (reviewing in) changes from a Github upstream project to Gerrit I'm hitting "the number of 
               pushed changes in a batch exceeds the max limit 10".
17:59 < kalle> This is mentioned and declined in https://phabricator.wikimedia.org/T200785
17:59 < kalle> Are there any recommendations on how to handle this?
18:00 < wikibugs> (PS2) Addshore: phpmetrics for Wikibase [integration/config] - https://gerrit.wikimedia.org/r/604760
18:00 < kalle> If I fetch --depth at the time, I'm hitting unlreated history problems instead.
18:01 < addshore> kalle: afaik that is all managed by the various permissions of groups on the repos
18:02 < addshore> hashar: can i build the jenkins job config things on jenkins itself yet? :D
18:03 < wikibugs> (PS1) Karl Wettin (WMSE): Edit Project Config [services/wikispeech/wikispeech_mockup] (refs/meta/config) - 
                  https://gerrit.wikimedia.org/r/604763
18:05 < wikibugs> Continuous-Integration-Config, Continuous-Integration-Infrastructure, Release-Engineering-Team, Pywikibot: 
                  Jenkins can't merge patchsets sometimes - https://phabricator.wikimedia.org/T252310 (Xqt) >>! In 
                  T252310#6215511, @Dvorapa wrote: >  > `stable` was suggested by several people to be made int...
18:06 <+hashar> addshore: I don't think you have the permission
18:06 <+hashar> kalle: yeah Gerrit has a hardset limit to prevent creating more than ten changes at a time
18:07 <+hashar> kalle: if importing from an external git repository, you want to just push to the branch
18:10 < wikibugs> Continuous-Integration-Config, Continuous-Integration-Infrastructure, Release-Engineering-Team, Pywikibot: 
                  Jenkins can't merge patchsets sometimes - https://phabricator.wikimedia.org/T252310 (Dvorapa) In standard 
                  git environemnt this would mean just to merge master branch into stable branch after...
18:10 < kalle> hashar: Our repo is an augmentation (blubber and ci-pipeline) of the upstream project with minor changes to 
               logs and what not. We'd really like to use review for these parts. But you suggest we switch to pushing? Not 
               sure I got what you wrote there.
18:11 <+hashar> kalle:  I need more context really
18:11 <+hashar> is that an upstream projects that you are trying to blubber / ci-pipeline ?
18:12 < paladox> kalle he means pushing straight into the repo without putting it up for review.
18:12 <+hashar> I am not sure it makes much sense in this context. There is probably a better way to craft a container from 
                an upstream project
18:12 <+hashar> or else
18:12 < paladox> git push origin HEAD:refs/heads/<branch> rather than git push origin HEAD:refs/for/<branch>
18:12 <+hashar> instead you could have our repo in Gerrit to have the upstream project as a git submodule
18:12 < kalle> hashar: Sure. There is an upstream project on Github. We use this in our project. So we have a downstream 
               repo in Gerrit in which we have a few minor changes, plus that we added Blubber stuff.
18:12 <+hashar> this way you separete the custom wikimedia stuff from the upstream code
18:13 <+hashar> ah
18:13  * hashar feels lagging today
18:13 < kalle> From time to time we want to merge in the upstream project to our downstream project.
18:13 <+hashar> ok ok
18:13 <+hashar> we do that for a bunch of projects. A common trend is in Gerrit to create an 'upstream' branch
18:13 <+hashar> then you would  push the change from github to that upstream branch
18:14 <+hashar> and later you can marge the updated upstream branch into your master branch and send that for review. That 
                will result in a single change
18:14 <+hashar> so something like:
18:14 <+hashar> git remote add origin <url of gerrit repo>
18:14 <+hashar> git remote add github <url of github repo>
18:14 <+hashar> git push github/master:gerrit:upstream
18:15 <+hashar> (that will send the upstream objects to gerrit without causing any change to be created in the Gerrit 
                interface)
18:15 <+hashar> cause it is a push to the 'upstream' branch
18:15 <+hashar> then
18:15 <+hashar> git checkout master
18:15 <+hashar> git merge upstream
18:15 <+hashar> (fix stuff)
18:15 <+hashar> and the result can be send to gerrit for review:  git push origin HEAD:refs/for/master
18:15 <+hashar> (or git-review)
18:16 <+hashar> which would just send the merg ecommit and result in a single change being created
18:17 < kalle> Alright, I'll try that out. Thanks for the help!
18:19 <+hashar> kalle: the idea is that wehn you send the upstream commits from github, Gerrit doesn't know about them and 
                would try to craft a change for each of them
18:19 < addshore> hashar: as I don't have any of the magic jjb stuff locally yet, and also can't use jenkins to build the 
                  thing itself, would you mind possibly deploying the job for me? :D
18:19 <+hashar> so surely if one import hundred of commits, there is no point in having a hundred of changes added to Gerrit 
                nor do we want to CR+2/Submit each of them ;]

Did a bit of digging and decoding and got the following to work

  • (one-off) Go to the mediawiki/services/wikispeech access panel and add a refs/heads/upstream section with the Push permission for mediawiki-services-wikispeech
  • for each repo
    1. git clone --origin gerrit <repo url> (with <repo url> being e.g. "ssh://lokal-profil@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech_mockup")
    2. cd <repo_name> (with <repo name> being e.g. "wikispeech_mockup")
    3. scp -p -P 29418 <gerrit username>@gerrit.wikimedia.org:hooks/commit-msg ".git/hooks/"
    4. git remote add github <github url> (with <github url> being e.g. "git@github.com:stts-se/wikispeech-server.git")
    5. Visit the gerrit access ui and create a branch named upstream using any commit before any of our changes as the initial revision (e.g. c5c118257cafce24774be5086c18525cb50020e1 for wikispeech_mockup)
    6. git fetch --all
    7. git push gerrit github/master:upstream

To push updates to master:

  1. git checkout master
  2. git merge gerrit/upstream
  3. git review

Todo:

Lokal_Profil changed the point value for this task from 8 to 1.

Change 610735 had a related patch set uploaded (by Lokal Profil; owner: Lokal Profil):
[mediawiki/services/wikispeech/wikispeech-server@refs/meta/config] Restore access settings to default

https://gerrit.wikimedia.org/r/610735

Change 610735 merged by Lokal Profil:
[mediawiki/services/wikispeech/wikispeech-server@refs/meta/config] Restore access settings to default

https://gerrit.wikimedia.org/r/610735

Change 611293 had a related patch set uploaded (by Karl Wettin (WMSE); owner: Karl Wettin (WMSE)):
[mediawiki/services/wikispeech/wikispeech_mockup@master] Rename Gerrit wikispeech_mockup repo to wikispeech-server

https://gerrit.wikimedia.org/r/611293

Change 611293 abandoned by Karl Wettin (WMSE):
[mediawiki/services/wikispeech/wikispeech_mockup@master] Rename Gerrit wikispeech_mockup repo to wikispeech-server

Reason:
Wrong repo #2

https://gerrit.wikimedia.org/r/611293

Change 612470 had a related patch set uploaded (by Karl Wettin (WMSE); owner: Karl Wettin (WMSE)):
[mediawiki/services/wikispeech/wikispeech_mockup@master] Rename Gerrit wikispeech_mockup repo to wikispeech-server

https://gerrit.wikimedia.org/r/612470

So this repository is weird.

TL;DR:

kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ git review -R
Creating a git remote called 'gerrit' that maps to:
	ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech_mockup.git
remote:
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again$ git clone "ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech-server" && scp -p -P 29418 karlwettin@gerrit.wikimedia.org:hooks/commit-msg "wikispeech-server/.git/hooks/"
Cloning into 'wikispeech-server'...
remote: Counting objects: 5, done
remote: Total 4102 (delta 0), reused 4102 (delta 0)
Receiving objects: 100% (4102/4102), 21.33 MiB | 4.79 MiB/s, done.
Resolving deltas: 100% (2785/2785), done.
commit-msg                                                                                                                                            100% 1806    15.4KB/s   00:00    
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again$ cd wikispeech-server/
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cat .git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech-server
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cat ~/opt/git-wmse.sh 
git config user.email "karl.wettin@wikimedia.se"
git config user.name "kalle"
git config gitreview.username "karlwettin"
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ git-wmse.sh 
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cat .git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech-server
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master
[user]
	email = karl.wettin@wikimedia.se
	name = kalle
[gitreview]
	username = karlwettin
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ git checkout -b rename-final-mockup-to-server
Switched to a new branch 'rename-final-mockup-to-server'
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cat .git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech-server
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master
[user]
	email = karl.wettin@wikimedia.se
	name = kalle
[gitreview]
	username = karlwettin
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cp ../../wikispeech
wikispeech-docker-compose/ wikispeech_mockup-server/  wikispeech-server/         
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cp ../../wikispeech-server/blubber-build.sh 
cp: missing destination file operand after '../../wikispeech-server/blubber-build.sh'
Try 'cp --help' for more information.
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cp ../../wikispeech-server/blubber-build.sh .
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cp ../../wikispeech-server/blubber-run.sh .
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cp ../../wikispeech-server/blubber-test.sh .
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ git add blubber-build.sh 
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ git add blubber-run.sh 
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ git add blubber-test.sh 
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ git commit
[rename-final-mockup-to-server e31917d] Rename Gerrit wikispeech_mockup repo to wikispeech-server
 3 files changed, 8 insertions(+), 8 deletions(-)
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cat .git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech-server
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master
[user]
	email = karl.wettin@wikimedia.se
	name = kalle
[gitreview]
	username = karlwettin
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ git review -R
Creating a git remote called 'gerrit' that maps to:
	ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech_mockup.git
remote: 
remote: Processing changes: new: 1 (\)
remote: Processing changes: new: 1 (|)
remote: Processing changes: refs: 1, new: 1 (|)
remote: Processing changes: refs: 1, new: 1 (|)
remote: Processing changes: refs: 1, new: 1, done            
remote: 
remote: SUCCESS        
remote: 
remote:   https://gerrit.wikimedia.org/r/c/mediawiki/services/wikispeech/wikispeech_mockup/+/612470 Rename Gerrit wikispeech_mockup repo to wikispeech-server [NEW]        
remote: 
To ssh://gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech_mockup.git
 * [new branch]      HEAD -> refs/for/master%topic=rename-final-mockup-to-server
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$ cat .git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech-server
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master
[user]
	email = karl.wettin@wikimedia.se
	name = kalle
[gitreview]
	username = karlwettin
[remote "gerrit"]
	url = ssh://karlwettin@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech_mockup.git
	fetch = +refs/heads/*:refs/remotes/gerrit/*
kalle@musa:~/projekt/wikimedia/wikispeech-services/post-server-repo/again/wikispeech-server$

Change 612470 abandoned by Karl Wettin (WMSE):
[mediawiki/services/wikispeech/wikispeech_mockup@master] Rename Gerrit wikispeech_mockup repo to wikispeech-server

Reason:
Wrong repo #3. See https://phabricator.wikimedia.org/T253851#6303340

https://gerrit.wikimedia.org/r/612470

Mine works fine. running git review (no -R). My config says

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "gerrit"]
	url = ssh://lokal-profil@gerrit.wikimedia.org:29418/mediawiki/services/wikispeech/wikispeech-server
	fetch = +refs/heads/*:refs/remotes/gerrit/*
[branch "master"]
	remote = gerrit
	merge = refs/heads/master
[remote "github"]
	url = git@github.com:stts-se/wikispeech-server.git
	fetch = +refs/heads/*:refs/remotes/github/*

Change 612656 had a related patch set uploaded (by Karl Wettin (WMSE); owner: Karl Wettin (WMSE)):
[mediawiki/services/wikispeech/wikispeech-server@master] Update repo in .gitreview

https://gerrit.wikimedia.org/r/612656

Change 612656 merged by Lokal Profil:
[mediawiki/services/wikispeech/wikispeech-server@master] Update repo in .gitreview

https://gerrit.wikimedia.org/r/612656

Change 612657 had a related patch set uploaded (by Karl Wettin (WMSE); owner: Karl Wettin (WMSE)):
[mediawiki/services/wikispeech/wikispeech-server@master] Rename Gerrit wikispeech_mockup repo to wikispeech-server

https://gerrit.wikimedia.org/r/612657

Change 612657 merged by Lokal Profil:
[mediawiki/services/wikispeech/wikispeech-server@master] Rename Gerrit wikispeech_mockup repo to wikispeech-server

https://gerrit.wikimedia.org/r/612657