A prerequisite to T85123 is probably to create a Jenkins instance that integrates with Phabricator and Harbormaster.
This assumes that Diffusion is already setup as a writable primary repository source. See: Gitblit-Deprecate
References:
http://www.guywarner.com/2014/05/integrating-jenkins-and-phabricator.html.
http://www.dctrwatson.com/2013/01/jenkins-and-phabricator/
See also: T31: [keyresult] Connect Differential code review with continuous integration
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Aklapper | T94265 Goal: Consolidate Phabricator as project management tool | |||
| Declined | None | T85123 Create a continuous integration plan for Wikimedia Phabricator patches | |||
| Duplicate | None | T89714 Integrate Jenkins with Phabricator with Harbormaster |
There is a big leap here in that Harbormaster is a prototype application for which upstream will not take feature or bug reports. We have thus far explicitly avoided putting ourselves in this position. To pursue we will probably first have to assign explicit WMF resources along with coordinating and very probably contracting upstream. In my my view, it is possible to maybe go this route without formally roping in upstream but will require more serious amounts of development and especially maintenance resources than we have seen since this began. It will also require any local patching to be rewritten as Harbormaster is fleshed out for phacility use cases. Which I think will happen relatively soon.
We definitely have a need for this or similar functionality, but as of now it would be a dead zone for support and is not possible for us to deploy it as anything other than a locked down experimental toy. It would be awesome if some serious resourcing conversations could take place soon.
I am not sure what Harbormaster provide and it probably does not match Zuul behavior ( I commented about it on T31#6408).
Imho the best way would be for Phabricator/Diffusion to emit events which we could teach Zuul to learn about. We will still need to adjust Zuul python code though but most of the logic is already there and well supported by upstream (they have, iirc, 7 people working full time on CI).
In this case it would be Differential or by proxy Harbormaster that would emit events so to speak in order to replicate our current workflow as closely as possible. In reality, the whole pipeline will have to be re-investigated I would imagine as it's going to be a lot of change form top to bottom with various pieces not fitting together in the same model as they do now. There are people doing jenkins type stuff without Harbormaster now with custom hooks into Differential. I'm not sure who upstream is here? Zuul? Either way, yeah this requires some concerted effort by some dedicated people.
I have suggested several times that we could keep gerrit as an upstream to differential: do primary code review in differential and when review is finished and submitted in phabricator we submit the patch upstream to gerrit to be merged. This should allow us to move incrementally to differential and it would give us a chance to play with harbormaster -> zuul integration without any big disruption to anyone's workflows. I discussed this with @hashar at the dev summit and he agreed. Antoine: Do you still think this would be a good way forward?
I like the idea of keeping the Gerrit as a middle war to smooth the transition. That is less disruption for the CI workflow and would let us switches teams one by one.
One feature we have is that people vote +2 to trigger the merge by Jenkins. We described it originally at http://www.mediawiki.org/wiki/Continuous_integration/Workflow and it should still be accurate.
As I understand it in Differential, some people are able to accept a patch and Differential does the actual merging. That is similar to people voting +2 and Zuul asking Gerrit to do the merge. I am not sure how we can mimic that part, we will need Differential to send a +2 event to Gerrit and thus trigger the merge via Zuul.
So as I understand it, the work would be to add a bunch of hooks in Differential to emit actions to Gerrit. Possibly via the REST API which should not be too hard. Should we start gathering elements and start writing some .plan ? If we could get a proof of concept by Lyon hackathon in May that would give us a great material to take a decision and finally start migrating.
Meanwhile, I am trying to get CI in isolated VMs.
For the records, Harbormaster is now un-prototyped in upstream (but work is ongoing in upstream T8089).