Adopting Zuul as a primary CI system for some GitLab hosted repos will afford us valuable gating and merging functionality (see T349872: Investigate and document "Depends-On" in GitLab), but we lose CI standardization and take on the burden of supporting (managing/documenting/implementing) an additional type of CI configuration.

If we promote the use of .zuul.yaml files in individual repos, we retain CI self service. However, it's an unknown whether or not the Ansible driven configuration of Zuul is sufficient for end users, and while GitLab CI has gone through community consultation, Zuul has not. If we manage the Zuul/Ansible configuration separately from end-user repos, we lose CI self service altogether. Neither of these options seem like great outcomes.

There may be an alternative, however, whereby we benefit from Zuul gating and retain CI self-service in the canonical form of .gitlab-ci.yml end-user configuration.

1. Integrate Zuul with GitLab for gating/merging.
   1. Zuul will continue to be the authoritative system that approves/unapproves/merges/comments on GitLab MRs.
   2. Zuul will continue to provision speculative repo state for the dependent and dependency repos.
2. Manage all (or most) Zuul configuration separately from individual repos in a Zuul trusted project, including Zuul tenants, pipelines, projects, and jobs.
3. Define a standard Zuul job for all GitLab repos (e.g. gitlab-pipeline) that:
   1. Creates a GitLab pipeline for the change (MR) using the GitLab API, passing the Zuul build UUID as a pipeline variable.
   2. Exposes the speculative repo state, which includes all dependent repos, to the GitLab CI jobs via something like rsyncd/ssh/git. A shared volume may also be possible, however such an implementation would have to be possible with both k8s and Docker based GitLab runners.
   3. Waits for the completion of the GL pipeline and fails/succeeds based on the result.
4. The repo-configured GitLab pipeline would in turn.
   1. Configure workflow rules such that only Zuul triggered pipelines are run (e.g. if: $CI_PIPELINE_SOURCE == 'api' && $ZUUL_BUILD)
   2. Sync the speculative repo state from the Zuul executor (/var/lib/zuul/builds) (or node /src) using rsync or git in either a before_script or hooks:pre_get_sources_script hook.
      • Note that if we use Git on the Zuul side and the pre_get_sources_script hook, we may be able to simply inject global Git insteadOf configuration which would save us redundant cloning operations and allow the GL jobs to function completely transparently with respect to the alternative Zuul Git source.
   3. Continue execution normally, and report ongoing and completion status to the associated MR.

Let's prove or disprove this concept.