Page MenuHomePhabricator

Investigate Kaniko as an option to build CI images
Closed, ResolvedPublic


Some things to consider:

  • Rootless/daemonless
  • Caching. Only remote is allowed; impact on performance needs verification
  • No support of Dockerfile frontends
  • Potentially limited implementation of low level features

Event Timeline

jnuche closed this task as Resolved.EditedMay 13 2022, 5:37 PM

Even though RelEng already decided in T307810 on using BuildKit as the way to build images in GitLab , I'm writing down the outcome of this story for documentation purposes.

A proof of concept can be found here. The Makefile and the manifests can be used to build a K8s cluster with a GitLab runner that generates images using Kaniko. A couple of branches in that repo demonstrate and compare the generation of images with and without remote caching.

When compared to the BuildKit solution:

  • Benefits:
    • Rootless/daemonless. No special privileges required and no possibility of building containers interfering with each other
    • Entire build process contained within the executor container. No need to maintain a separate build environment and no need to secure access to it
    • Management of remote cache layers via TTLs
  • Downsides:
    • No support for Dockerfile frontends
    • No local caching (except for base images)
    • No multi-architecture support. Although in our case we probably don't need this
    • Re-implementation of low-level features. In the best-case scenario, some features already in BuildKit will take longer to be implemented
    • (Potentially) Access to credentials to publish production images is required. But this wouldn't be an issue for non-prod images or when using a proxy registry