@Dzahn we have a repo in Diffusion ({rWRES}) that mirrors a Github repo. Is it possible to deploy it, rather than a Gerrit repo, at research.wikimedia.org? What are the missing pieces needed for deployment?

@bmansurov It's possible, as in the git::clone puppet class has a source option "phabricator", but it's not advised because migrating code review to phabricator has been declined in T119908.

It's technically easy to switch though, just a one-line patch changing the source and not against a policy. You might also want to consult release engineering. (cc: @demon )

@Dzahn, thanks! We'll be doing code review in Github, and the Diffusion mirror is there only because we want to deploy.

@demon, do you see any problems with this approach, i.e. mirroring a Github repo to Diffusion and deploying code from Diffusion?

@demon your input would be appreciated. Thanks!

Github -> mirror to Diffusion -> deploy with puppet's ensure => latest is effectively deploying from Github, which we don't do. One of two things needs to happen as a gateway....

1. We don't use ensure => latest in puppet, but rather a tag or sha1 to be explicit. This would require a puppet change on each update.
2. We keep ensure => latest, but the mirroring to Diffusion isn't automatic.

In Design's case, we went with option (2) a few weeks ago. They push to Gerrit when they want to deploy, and puppet ensures the latest copy is pulled.
Either is fine by me, but (2) is probably less effort for your team for deploying.

PS: There's always option (3) of deploy via scap, in which case you can use Diffusion for mirroring automatically, you'd just need to scap deploy the correct versions from the deploy masters.

For ORES deployments, we use submodules to specify the exact sha1 we expect to pull from (github-->diffusion). The main repository (that manages said submodules) is a regular ol' gerrit repo. Here's what things look like:

• ores-deploy (gerrit)
  • ores (github --> diffusion via sha1)
  • editquality (github --> diffusion via sha1)
  • articlequality (github --> diffusion via sha1)
  • draftquality (github --> diffusion via sha1)
  • ores-wheels (gerrit)

Every time we update ores-deploy by pulling in a submodule, that triggers a gerrit review cycle and references an explicit sha1. It's been a pretty straightforward workflow for us.

Thanks both for comments.

@Halfak, so it looks like Github -> Gerrit cannot be automatic with this setup, am I right? For example, in this patch, Adam had to manually update submodules. What we're after is that we want Github merges to deploy automatically without any manual work.

Right. I think the manual submodule pattern is very convenient though. E.g. we keep configuration for our specific deployment (ores-deploy) in gerrit with the service module (ores) as a submodule. Very often, deployments are just config changes and periodically we bump the submodules to a specific version.

But indeed it seems that github --> prod without any other work is one of the things we want to avoid.

Thanks, @Halfak!

@DarTar to sum up our conversation so far, we can accept Github pull requests but changes need to be manually deployed. On the other hand, in our current system, once we merge a patch in Gerrit, it will appear immediately on the site. Given this information, what do you prefer we do — keep the current workflow or switch to accepting Github pull requests and manually deploying them?

We discussed T187957#4146727 with Dario, and decided to keep things as is for now. We can create changes both on Gerrit and Github. Changes created in Gerrit will be merged using the Gerrit workflow (+2'ing). Changes created on Github will be pushed to Gerrit manually.