Implement a way to bring GitHub pull requests into gerrit
OpenPublic

Description

Author: sumanah

Description:
First: Find the various GitHub mirrors of MediaWiki. Turn one into an up-to-date clone of our codebase.

Next, harder step: Make an interface to accept GitHub pull requests as merge requests in Gerrit, or do two-way syncing automatically via a bot.

Erik: "although I'm guessing that falls into the "hard" category, it could give us huge wins in terms of casual contribution.

Chad: "Yeah, that's definitely not straightforward--would need some careful thought to make sure we're doing it in a way that makes sense on our end too."


Version: unspecified
Severity: enhancement

bzimport added a subscriber: wikibugs-l.
bzimport set Reference to bz35497.
bzimport created this task.Via LegacyMar 26 2012, 6:36 PM
bzimport added a comment.Via ConduitMar 26 2012, 6:38 PM

sumanah wrote:

Chad, assigning this to you since you're the migration lead, but if you do not think you can start on it in the next month, please de-assign from yourself. Thanks.

demon added a comment.Via ConduitMar 26 2012, 6:42 PM

Let's make this bug about pulling things back from outside repos into our ecosystem. Mirroring our repo to GitHub/Gitorious is much easier and already has bug 35429.

RyanLane added a comment.Via ConduitMar 27 2012, 3:21 AM

Please make sure to be careful when setting up replication to github. We have private repos, and it would suck to have it replicated to github accidentally.

hashar added a comment.Via ConduitMar 27 2012, 7:07 PM

What is the point in replicating to github if we are not going to use it to handle pull requests / issue tracking. I think it will confuse people that will submit patches there that we will probably never take care of.

I would prefer we setup a Gitorious instance instead carefully restricted to hide / disallow pulls and issues tracking.

bzimport added a comment.Via ConduitMar 27 2012, 7:10 PM

sumanah wrote:

We can probably do something to indicate "sorry, please don't submit issues here", but we do want people to submit pull requests to GitHub *and then actually pull them into Gerrit*. Lots of casual developers are on GitHub and are accustomed to working there -- we should be there too to take advantage of that.

demon added a comment.Via ConduitMar 27 2012, 10:45 PM

(In reply to comment #4)

What is the point in replicating to github if we are not going to use it to
handle pull requests / issue tracking. I think it will confuse people that
will submit patches there that we will probably never take care of.

You can disable issue tracking on GitHub, iirc.

In any case, turning pull requests from GitHub/Gitorious into merge requests for Gerrit is not the simplest task and I don't see it happening in the very immediate future.

MarkAHershberger added a comment.Via ConduitMar 28 2012, 7:39 PM

(In reply to comment #6)

In any case, turning pull requests from GitHub/Gitorious into merge requests
for Gerrit is not the simplest task and I don't see it happening in the very
immediate future.

If I can find an api to list merge requests on GitHub and Gitorious, this shouldn't be a problem.

Eloquence added a comment.Via ConduitMar 28 2012, 7:43 PM

I agree it's probably tricky, but it would be awesome. GitHub has a huge existing developer community, lots of people are familiar with it, and our current account creation process is a higher barrier to entry. So let's please look into this when we have time.

Krinkle added a comment.Via ConduitMar 29 2012, 4:02 AM

Making this bug more specific about brining Github pull requests into Gerrit.

The mirroring in general would be one way (master) and is bug 22596.

bzimport added a comment.Via ConduitApr 3 2012, 11:03 PM

sumanah wrote:

Krinkle said:

"PHP has migrated to git at March 19 http://www.mail-archive.com/internals@lists.php.net/msg57098.html contains some links and to their wiki page of gitworkflow...

"They don't use github as primary source, as a one-way mirror only. however they do take pull-requests from there (since github is just git underneath, the php devs can merge from a github pull request into their repo, just like we can merge a github PR into gerrit for example."

Eloquence added a comment.Via ConduitApr 23 2012, 9:52 PM

Doesn't solve this particular problem, but Jonathan Leto, who's explored GitHub/Gerrit integration before, sent me a link to an automatic pull request closer which might come in handy as reference code for anyone wanting to work on actual pull request integration.

https://github.com/cloudfoundry/oss-tools/tree/master/pr-bounce

Krinkle added a comment.Via ConduitJul 5 2012, 10:32 AM

So, some ideas:

  • Don't use github.com/mediawiki/core as name
    • means we duplicate user groups/rights with the org "Wikimania" at github
    • means we're going to promote ourselves as "core". e.g. github.com/Krinkle/core.
    • any advantages? hashar mentioned we want to be able to allow volunteers to help out with the handling of pull-request and that to grant them rights we'd want to have it outside the @wikimedia organization. However this is not needed because:
      • github allows collaboration without any rights at all. You can leave inline comments, and stuff on any pull request anywhere
      • as in comment 12, they are pull-able in many different formats including plain "git pull" so anyone can pull it locally, and if they have a labs ldap account they can also push straight to gerrit for review.
      • github supports auto-closing of pull-requests when the commit hash is pushed into the repo, so no maintenance there either. And if the commit has to be amended, including "fixes GH-123" or "closes GH-123" will also close the relevant PR as soon as it is merged. And if all else fails, a repo collab in @wikimedia (of which there are many[1]) can just push "Close" manually on github So volunteers have complete access without needing to be manually added to anything, this is what made GitHub works. And if its only about closing some exceptional ones, then I'm sure we'll manage that.
  • Use github.com/wikimedia/mediawiki-core
    • ideally we'd have some kind of auto-push from gerrit or jenkins, otherwise just set up a 30 minute cron somewhere to pull gerrit -f and push github -f.
    • admin settings: pull-request: true, issues: false, wiki: false

[1] https://github.com/wikimedia

Krinkle added a comment.Via ConduitJul 5 2012, 10:36 AM

Note that setting up of the mirror in the first place is bug 38196.

Aklapper added a comment.Via ConduitOct 15 2012, 3:29 PM

(In reply to comment #4)

What is the point in replicating to github if we are not going to use it to
handle pull requests / issue tracking. I think it will confuse people

(In reply to comment #5)

We can probably do something to indicate "sorry, please don't submit issues
here", but we do want people to submit pull requests to GitHub

FYI, according to https://github.com/blog/1184-contributing-guidelines you can clarify contribution guidelines for people who submit pull requests or issues.

demon added a comment.Via ConduitOct 15 2012, 3:44 PM

(In reply to comment #15)

(In reply to comment #4)
> What is the point in replicating to github if we are not going to use it to
> handle pull requests / issue tracking. I think it will confuse people

(In reply to comment #5)
> We can probably do something to indicate "sorry, please don't submit issues
> here", but we do want people to submit pull requests to GitHub

FYI, according to https://github.com/blog/1184-contributing-guidelines you can
clarify contribution guidelines for people who submit pull requests or issues.

Oh, didn't know that. We could easily add a CONTRIBUTING file to mediawiki core :)

Krinkle added a comment.Via ConduitMar 5 2013, 8:01 PM

The Issues and Wiki features in GitHub repositories can be disabled easily. There is no need for any textual guideline to enforce.

We might want to have contribution guidelines for other reasons, but not this.

Krinkle added a comment.Via ConduitMar 5 2013, 8:04 PM

(In reply to comment #5)

We can probably do something to indicate "sorry, please don't submit issues
here", but we do want people to submit pull requests to GitHub

(In reply to comment #6)

You can disable issue tracking on GitHub, iirc.

(In reply to comment #13)

  • admin settings: pull-request: true, issues: false, wiki: false

(In reply to comment #17)

The Issues and Wiki features in GitHub repositories can be disabled easily.

bug 38196 has been fixed. We have replication now, and the issue tracker and wiki are disabled:

https://github.com/wikimedia/mediawiki-core

Qgil added a comment.Via ConduitApr 9 2013, 5:53 PM

In case it helps, I have listed this task at

https://www.mediawiki.org/wiki/Project:New_contributors#Merging_GitHub_pull_requests

What help do you need to get it done?

Also a question: is this about bringing pull requests to Gerrit as soon as they are submitted or about merging the ones that the maintainers accept in GitHub? The difference is where is the discussion supposed to happen.

bzimport added a comment.Via ConduitApr 9 2013, 6:01 PM

sumanah wrote:

http://lists.wikimedia.org/pipermail/wikitech-l/2013-March/067665.html ("Moving a GitHub Pull Request to Gerrit Changeset manually") and the subsequent thread may be useful.

yuvipanda added a comment.Via ConduitApr 15 2013, 7:33 AM

I'm working on automating this and should have something in a day or so. See https://mediawiki.org/wiki/User:Yuvipanda/G2G and https://github.com/yuvipanda/SuchABot

Qgil added a comment.Via ConduitMay 23 2013, 6:05 PM

Yuvi's tools are now bridging GitHub and Gerrit for a number of repostories that have requested it. More details at https://www.mediawiki.org/wiki/User:Yuvipanda/G2G

Seems to work. FIXED?

yuvipanda added a comment.Via ConduitMay 24 2013, 6:45 AM

It is currently enabled for the following repositories, after consulting with people responsible for the repos:

qa/browsertests
extensions/MobileFrontend
extensions/PostEdit
extensions/GuidedTour
extensions/EventLogging
extensions/GettingStarted

Any pull requests to the appropriate repo in GitHub will automatically (and instantly!) have a Gerrit patchset created, and a comment will be left on the GitHub Pull Request informing people of the appropriate Gerrit URL. This is currently running on Wikimedia Tools Labs, and should be considered experimental. If anyone else wants to have their repo added to the 'watched' list, do let me know :)

Qgil added a comment.Via ConduitMay 24 2013, 7:08 PM

Resolving as FIXED. There is a functional proof of concept and some precedents. There is a process to request new repos to be added. And there might be bugs, but we have the usual Bugzilla process for this.

Thank you to everybody involved! This is just the beginning.

yuvipanda added a comment.Via ConduitJun 16 2013, 4:35 PM

It now syncs Comments / Changeset Abandon / Merging / Restore on Gerrit on GitHub! See https://github.com/wikimedia/apps-android-commons/pull/6 for an example (though usually it closes the PR automatically).

It doesn't sync inline comments yet, so I'll need to figure that out. There is also a Redis based reliable queue running on Tools-labs, so future tools that depend on Gerrit can be easily built too. Ideas for such welcome :)

cscott added a comment.Via ConduitAug 14 2013, 2:50 PM

Yuvi, what's the process for adding new repos? Maybe some docs can be added to http://www.mediawiki.org/wiki/Gerrit (which links to this bug).

Krenair added a comment.Via ConduitMar 7 2014, 8:05 PM

I guess it's time to reopen this. The bot has been down for a long while and Yuvi doesn't know when he's going to get time to fix it.

bzimport added a comment.Via ConduitSep 23 2014, 7:30 PM

sumanah wrote:

If Yuvi does not have time to do this (and, presumably, to get GitHub->Phabricator syndication going), does anyone else?

Qgil added a comment.Via ConduitSep 23 2014, 7:53 PM

If Phabricator is the focus of this report now, then we will have a duplicate after the migration:

GitHub->Phabricator bridge for new contributors
https://phabricator.wikimedia.org/T173

Consolidating discussion in T173 - please comment there.

Spage changed the title from "Implement a way to bring GitHub pull requests into Phabricator" to "Implement a way to bring GitHub pull requests into gerrit".Via WebMar 26 2015, 9:15 PM
Spage reopened this task as "Open".
Spage lowered the priority of this task from "High" to "Normal".
Spage set Security to None.
Spage added a subscriber: Spage.Via WebMar 26 2015, 9:22 PM

This bug from 2012 was about GitHub -> Gerrit. Someone mistakenly changed it to GitHub -> Phabricator and made it a dupe of T173. But the Gerrit integration described in T37497#398909 is no longer working, and it's still nice to have regardless of what happens with Phabricator. YuviPanda commented on IRC

I have a bot that does that but it has been dead for years. Takes maybe a day of work to bring back up but I haven't had that day. Hackathon project maybe

Ricordisamoa added a subscriber: Ricordisamoa.Via WebTue, May 12, 11:48 AM

Add Comment