Release Engineering's "GitLab-a-thon" sprint for May 10th-24th (roughly) focused on the mechanics of migrating a Wikimedia service to GitLab, setting up a CI pipeline, building container images from that service, and publishing images to the Wikimedia registry. We selected the Blubber project as a good candidate for experimentation:

We evaluated build mechanisms including GitLab's suggested docker-in-docker, Kaniko, Podman, and BuildKit:

We ultimately landed on BuildKit as the least constraining for future options, and the most in line with features we'd like to offer.

We explored a range of options for building and publishing, including variations on:

  • Building on runners provisioned on a DigitalOcean Kubernetes cluster and importing to the production registry from some trusted location (contint, for example) by way of a shim.
  • Building on trusted runners and publishing to the GitLab Container Registry, then importing to the production registry by way of a shim.
  • Building on trusted runners and publishing directly from there to the prod registry, authenticated against GitLab by way of JWT.

We eventually landed on this latter, and work is well underway on implementation: T308501: Authenticate trusted runners for registry access against GitLab using temporary JSON Web Token

Other work included implementing CI for Blubber on GitLab (T307534), improvements to user-facing documentation (T307535, T307538), enforcing the allowlist for container images in GitLab CI (T291978), experimentation with the GitLab Container Registry (T307537), and extensive discussions with ServiceOps on GitLab infrastructure.

Written by brennen on May 31 2022, 6:30 PM.
Software Engineer, Release Engineering
"Like" token, awarded by sbassett."Party Time" token, awarded by kostajh."Meh!" token, awarded by bd808."Yellow Medal" token, awarded by thcipriani.

Event Timeline