Implement a way to bring GitHub pull requests into gerrit
Open, LowPublic

Description

Author: sumanah

Description:
First: Find the various GitHub mirrors of MediaWiki. Turn one into an up-to-date clone of our codebase.

Next, harder step: Make an interface to accept GitHub pull requests as merge requests in Gerrit, or do two-way syncing automatically via a bot.

Erik: "although I'm guessing that falls into the "hard" category, it could give us huge wins in terms of casual contribution.

Chad: "Yeah, that's definitely not straightforward--would need some careful thought to make sure we're doing it in a way that makes sense on our end too."


Version: unspecified
Severity: enhancement

bzimport set Reference to bz35497.
bzimport added a subscriber: Unknown Object (MLST).
bzimport created this task.Mar 26 2012, 6:36 PM

sumanah wrote:

Chad, assigning this to you since you're the migration lead, but if you do not think you can start on it in the next month, please de-assign from yourself. Thanks.

demon added a comment.Mar 26 2012, 6:42 PM

Let's make this bug about pulling things back from outside repos into our ecosystem. Mirroring our repo to GitHub/Gitorious is much easier and already has bug 35429.

Please make sure to be careful when setting up replication to github. We have private repos, and it would suck to have it replicated to github accidentally.

What is the point in replicating to github if we are not going to use it to handle pull requests / issue tracking. I think it will confuse people that will submit patches there that we will probably never take care of.

I would prefer we setup a Gitorious instance instead carefully restricted to hide / disallow pulls and issues tracking.

sumanah wrote:

We can probably do something to indicate "sorry, please don't submit issues here", but we do want people to submit pull requests to GitHub *and then actually pull them into Gerrit*. Lots of casual developers are on GitHub and are accustomed to working there -- we should be there too to take advantage of that.

(In reply to comment #4)

What is the point in replicating to github if we are not going to use it to
handle pull requests / issue tracking. I think it will confuse people that
will submit patches there that we will probably never take care of.

You can disable issue tracking on GitHub, iirc.

In any case, turning pull requests from GitHub/Gitorious into merge requests for Gerrit is not the simplest task and I don't see it happening in the very immediate future.

(In reply to comment #6)

In any case, turning pull requests from GitHub/Gitorious into merge requests
for Gerrit is not the simplest task and I don't see it happening in the very
immediate future.

If I can find an api to list merge requests on GitHub and Gitorious, this shouldn't be a problem.

I agree it's probably tricky, but it would be awesome. GitHub has a huge existing developer community, lots of people are familiar with it, and our current account creation process is a higher barrier to entry. So let's please look into this when we have time.

Making this bug more specific about brining Github pull requests into Gerrit.

The mirroring in general would be one way (master) and is bug 22596.

sumanah wrote:

Krinkle said:

"PHP has migrated to git at March 19 http://www.mail-archive.com/internals@lists.php.net/msg57098.html contains some links and to their wiki page of gitworkflow...

"They don't use github as primary source, as a one-way mirror only. however they do take pull-requests from there (since github is just git underneath, the php devs can merge from a github pull request into their repo, just like we can merge a github PR into gerrit for example."

Doesn't solve this particular problem, but Jonathan Leto, who's explored GitHub/Gerrit integration before, sent me a link to an automatic pull request closer which might come in handy as reference code for anyone wanting to work on actual pull request integration.

https://github.com/cloudfoundry/oss-tools/tree/master/pr-bounce

So, some ideas:

  • Don't use github.com/mediawiki/core as name
    • means we duplicate user groups/rights with the org "Wikimania" at github
    • means we're going to promote ourselves as "core". e.g. github.com/Krinkle/core.
    • any advantages? hashar mentioned we want to be able to allow volunteers to help out with the handling of pull-request and that to grant them rights we'd want to have it outside the @wikimedia organization. However this is not needed because:
      • github allows collaboration without any rights at all. You can leave inline comments, and stuff on any pull request anywhere
      • as in comment 12, they are pull-able in many different formats including plain "git pull" so anyone can pull it locally, and if they have a labs ldap account they can also push straight to gerrit for review.
      • github supports auto-closing of pull-requests when the commit hash is pushed into the repo, so no maintenance there either. And if the commit has to be amended, including "fixes GH-123" or "closes GH-123" will also close the relevant PR as soon as it is merged. And if all else fails, a repo collab in @wikimedia (of which there are many[1]) can just push "Close" manually on github So volunteers have complete access without needing to be manually added to anything, this is what made GitHub works. And if its only about closing some exceptional ones, then I'm sure we'll manage that.
  • Use github.com/wikimedia/mediawiki-core
    • ideally we'd have some kind of auto-push from gerrit or jenkins, otherwise just set up a 30 minute cron somewhere to pull gerrit -f and push github -f.
    • admin settings: pull-request: true, issues: false, wiki: false

[1] https://github.com/wikimedia

Note that setting up of the mirror in the first place is bug 38196.

(In reply to comment #4)

What is the point in replicating to github if we are not going to use it to
handle pull requests / issue tracking. I think it will confuse people

(In reply to comment #5)

We can probably do something to indicate "sorry, please don't submit issues
here", but we do want people to submit pull requests to GitHub

FYI, according to https://github.com/blog/1184-contributing-guidelines you can clarify contribution guidelines for people who submit pull requests or issues.

demon added a comment.Oct 15 2012, 3:44 PM

(In reply to comment #15)

(In reply to comment #4)
> What is the point in replicating to github if we are not going to use it to
> handle pull requests / issue tracking. I think it will confuse people

(In reply to comment #5)
> We can probably do something to indicate "sorry, please don't submit issues
> here", but we do want people to submit pull requests to GitHub

FYI, according to https://github.com/blog/1184-contributing-guidelines you can
clarify contribution guidelines for people who submit pull requests or issues.

Oh, didn't know that. We could easily add a CONTRIBUTING file to mediawiki core :)

The Issues and Wiki features in GitHub repositories can be disabled easily. There is no need for any textual guideline to enforce.

We might want to have contribution guidelines for other reasons, but not this.

(In reply to comment #5)

We can probably do something to indicate "sorry, please don't submit issues
here", but we do want people to submit pull requests to GitHub

(In reply to comment #6)

You can disable issue tracking on GitHub, iirc.

(In reply to comment #13)

  • admin settings: pull-request: true, issues: false, wiki: false

(In reply to comment #17)

The Issues and Wiki features in GitHub repositories can be disabled easily.

bug 38196 has been fixed. We have replication now, and the issue tracker and wiki are disabled:

https://github.com/wikimedia/mediawiki-core

Qgil added a comment.Apr 9 2013, 5:53 PM

In case it helps, I have listed this task at

https://www.mediawiki.org/wiki/Project:New_contributors#Merging_GitHub_pull_requests

What help do you need to get it done?

Also a question: is this about bringing pull requests to Gerrit as soon as they are submitted or about merging the ones that the maintainers accept in GitHub? The difference is where is the discussion supposed to happen.

sumanah wrote:

http://lists.wikimedia.org/pipermail/wikitech-l/2013-March/067665.html ("Moving a GitHub Pull Request to Gerrit Changeset manually") and the subsequent thread may be useful.

I'm working on automating this and should have something in a day or so. See https://mediawiki.org/wiki/User:Yuvipanda/G2G and https://github.com/yuvipanda/SuchABot

Qgil added a comment.May 23 2013, 6:05 PM

Yuvi's tools are now bridging GitHub and Gerrit for a number of repostories that have requested it. More details at https://www.mediawiki.org/wiki/User:Yuvipanda/G2G

Seems to work. FIXED?

It is currently enabled for the following repositories, after consulting with people responsible for the repos:

qa/browsertests
extensions/MobileFrontend
extensions/PostEdit
extensions/GuidedTour
extensions/EventLogging
extensions/GettingStarted

Any pull requests to the appropriate repo in GitHub will automatically (and instantly!) have a Gerrit patchset created, and a comment will be left on the GitHub Pull Request informing people of the appropriate Gerrit URL. This is currently running on Wikimedia Tools Labs, and should be considered experimental. If anyone else wants to have their repo added to the 'watched' list, do let me know :)

Qgil added a comment.May 24 2013, 7:08 PM

Resolving as FIXED. There is a functional proof of concept and some precedents. There is a process to request new repos to be added. And there might be bugs, but we have the usual Bugzilla process for this.

Thank you to everybody involved! This is just the beginning.

It now syncs Comments / Changeset Abandon / Merging / Restore on Gerrit on GitHub! See https://github.com/wikimedia/apps-android-commons/pull/6 for an example (though usually it closes the PR automatically).

It doesn't sync inline comments yet, so I'll need to figure that out. There is also a Redis based reliable queue running on Tools-labs, so future tools that depend on Gerrit can be easily built too. Ideas for such welcome :)

Yuvi, what's the process for adding new repos? Maybe some docs can be added to http://www.mediawiki.org/wiki/Gerrit (which links to this bug).

I guess it's time to reopen this. The bot has been down for a long while and Yuvi doesn't know when he's going to get time to fix it.

sumanah wrote:

If Yuvi does not have time to do this (and, presumably, to get GitHub->Phabricator syndication going), does anyone else?

Qgil added a comment.Sep 23 2014, 7:53 PM

If Phabricator is the focus of this report now, then we will have a duplicate after the migration:

GitHub->Phabricator bridge for new contributors
https://phabricator.wikimedia.org/T173

Consolidating discussion in T173 - please comment there.

Spage changed the title from "Implement a way to bring GitHub pull requests into Phabricator" to "Implement a way to bring GitHub pull requests into gerrit".Mar 26 2015, 9:15 PM
Spage reopened this task as "Open".
Spage lowered the priority of this task from "High" to "Normal".
Spage set Security to None.
Spage added a subscriber: Spage.Mar 26 2015, 9:22 PM

This bug from 2012 was about GitHub -> Gerrit. Someone mistakenly changed it to GitHub -> Phabricator and made it a dupe of T173. But the Gerrit integration described in T37497#398909 is no longer working, and it's still nice to have regardless of what happens with Phabricator. YuviPanda commented on IRC

I have a bot that does that but it has been dead for years. Takes maybe a day of work to bring back up but I haven't had that day. Hackathon project maybe

greg lowered the priority of this task from "Normal" to "Low".Sep 10 2015, 11:05 PM

Add Comment