Page MenuHomePhabricator

Use Diffusion as canonical location for browsing code repos (not gitblit)
Closed, ResolvedPublic

Description

Mirroring all Gerrit repos in Phabricator with Diffusion is a goal in itself, not directly tied to #code-review , that would allow us to deprecate Gitblit (http://git.wikimedia.org/).

We need to make sure that there is a way to turn these mirrored repositories into Phabricator local repositories, in order to allow a smooth future migration from Gerrit.

@demon says:

I'm mainly concerned about stable URL structures and preserving references when we move a repo from being hosted in Gerrit to being hosted in Phabricator.

Now that all repos are publicly available, update:

  • all template links to GitHub for core (easy, same URL structure: {{MW file}} done, ...)
  • same, for extensions (how to get callsigns?)
  • non-template links to GitHub for core (Manual:Code done, ...)
  • non-template links to GitHub for extensions

Links to gitblit will be handled with redirects, see the specific task for that.

Event Timeline

Qgil raised the priority of this task from to Low.
Qgil updated the task description. (Show Details)
Qgil changed Security from none to None.
Qgil added subscribers: Qgil, MZMcBride, demon.

Enabling Diffusion and mirroring a couple of Wikimedia is simple. This had been done in the old good fab, and it could be done right now at https://phab-01.wmflabs.org/diffusion/

However, how should we mirror all the Gerrit repositories? With a script?

demon raised the priority of this task from Low to Medium.
demon added a project: MediaWiki-Core-Team.
This comment was removed by demon.

I think we'll have to do the repositories one by one with a signup process. We need to pick names for the repos on Phabricator (they don't have to be the same, and I don't think we're going to replicate our tree-ish structure), plus callsigns. Both of these are permanent once we stop mirroring and start hosting, so they need to be chosen well.

Once imported, Phabricator will keep them up to date on its own.

In T752#12599, @Chad wrote:

I think we'll have to do the repositories one by one with a signup process.

Alright, then this probably invalidates this other statement?

In T616#12534, @Qgil wrote:

We should probably check first if there are any performance issues before just plunging on ahead

Of course. Something like this should be deployed first on phab-01 or another Labs instance for testing.

If this is in fact manual work, we cannot do it twice. Unless there is a way to include all this information in the Phabricator labs Puppet rules?

In T752#12673, @Qgil wrote:
In T616#12534, @Qgil wrote:

We should probably check first if there are any performance issues before just plunging on ahead

Of course. Something like this should be deployed first on phab-01 or another Labs instance for testing.

If this is in fact manual work, we cannot do it twice. Unless there is a way to include all this information in the Phabricator labs Puppet rules?

My comment was about the use of Diffusion as a replacement for GitBlit (read-only Web browsing of repos), and that we should quickly sanity-check performance before just doing it for all repos. However, you can have lots of read-only copies of a repo, so it's not urgent to switch GitBlit off. We can do it a few dozen times (but let's not).

In T752#12673, @Qgil wrote:
In T752#12599, @Chad wrote:

I think we'll have to do the repositories one by one with a signup process.

Alright, then this probably invalidates this other statement?

No, this comment is about switching to Diffusion as the replacement for Gerrit (master server for a repo, in which it is changed). Because you can only ever have one master, this means a breaking change:

  • manually disabling write in Gerrit for the repo
  • manually(?) enabling write in Diffusion for the repo
  • getting everyone using the repo to change their remote repo configuration
  • updating all the documentation

This is both tedious and not something we can rush, but once we've got a mostly-agreed workflow process in place (including CI integration working) we will be able to do this for a few isolated repos to verify it works, and then at some point throw the switch and make everyone switch.

In T752#12673, @Qgil wrote:
In T616#12534, @Qgil wrote:

We should probably check first if there are any performance issues before just plunging on ahead

Of course. Something like this should be deployed first on phab-01 or another Labs instance for testing.

If this is in fact manual work, we cannot do it twice. Unless there is a way to include all this information in the Phabricator labs Puppet rules?

My comment was about the use of Diffusion as a replacement for GitBlit (read-only Web browsing of repos), and that we should quickly sanity-check performance before just doing it for all repos. However, you can have lots of read-only copies of a repo, so it's not urgent to switch GitBlit off. We can do it a few dozen times (but let's not).

No, we can't do it a few dozen times I think. The repositories can be renamed easily enough in Phabricator (as can their clone paths). The callsign is forever and once URLs start building on them I wouldn't want to delete them for a manual rename.

In T752#12673, @Qgil wrote:
In T752#12599, @Chad wrote:

I think we'll have to do the repositories one by one with a signup process.

Alright, then this probably invalidates this other statement?

No, this comment is about switching to Diffusion as the replacement for Gerrit (master server for a repo, in which it is changed). Because you can only ever have one master, this means a breaking change:

  • manually disabling write in Gerrit for the repo
  • manually(?) enabling write in Diffusion for the repo
  • getting everyone using the repo to change their remote repo configuration
  • updating all the documentation

This is both tedious and not something we can rush, but once we've got a mostly-agreed workflow process in place (including CI integration working) we will be able to do this for a few isolated repos to verify it works, and then at some point throw the switch and make everyone switch.

We're generally in agreement here. Except I also meant it for Gitblit replacement per above.

We could automate the process to do in batches or something but we can't do it until we're settled on names for individual repos.

Yes, it's also going to identify abandoned projects ;-)

Importing MediaWiki now since it's the one we're sure won't be namespaced and we're clear on the callsign for. Once it's working I'll open it up for viewing.

I like that top bar search by hash (e.g. 00cf19dc65) works, seemingly everywhere.

greg renamed this task from Wikimedia code repository browser in Phabricator to Use Diffusion as canonical location for browsing code repos (not gitblit).Nov 6 2015, 11:44 PM
greg updated the task description. (Show Details)