Page MenuHomePhabricator

Replace toollabs-docker-images with cloud-services-buildpacks
Closed, DeclinedPublic

Description

On investigating buildpacks a bit in pockets of spare time, one of the simplest ways to start using them to promote understanding within the team and better handle user requests in Toolforge would be to begin the process of replacing the management of docker images for Toolforge Kubernetes with buildpack-generated images.

This would basically look like taking the python3.7-sssd image (for instance) and dropping it from toollabs-docker-images while establishing the buildpack for it, along with appropriate scripts to simplify rebuilds of the pack.

This generally would introduce the complexity of having a second script to run to rebuild docker images, but that would only be true until all legacy images are now buildpacks. Besides that, this would make eventually migrating to CD-controlled buildpacks vastly simpler.

This is expressly not introducing the functionality of pack to Toolforge to dynamically generate the image based on the buildpacks because that needs to happen in a closed-loop environment like CD. This is simply about changing tooling around images to one that is a bit more flexible and potentially a lot less "OS-based" since buildpack-inspired processes like the official Python images on Docker hub and buildpack stacks in general, usually start from a tar file of the runtime built by the builder rather than OS packages.

Event Timeline

Bstorm created this task.

Examples to happen once they work locally and after that a new repo is requested.

Probably the coolest feature this would add https://buildpacks.io/docs/concepts/operations/rebase/
Other parts are not nearly as cool are are mostly a win for future plans. The buildpacks would have excellent deduplication in terms of storage when people are using them in a more direct push-to-deploy fashion, but we'd still have at least one layer of image for each tool on top of the stack images. In general, that really might not be hard to corral into a Ceph setup, though.

In this phase, it would look more like establishing a set of stacks with builders configured to produce images that aim to talk to NFS for actual runtime stuff. Strangely, that's less efficient in terms of total storage (because of poor deduplication of NFS), but it is how this task will look when complete.

Splitting off tidbits of the toollabs-webservice package (webservice-runner) may be part of this task as well so that it is easier to include directly in the buildpack instead of installing a package.

I think this is technically doable, but given how different the buildpack images are turning out to be (non-root language installs, different directory layout), I don't think we could safely replace the current images with buildpack based ones without requiring some intervention from users.

The buildpack images are going to be fundamentally incompatible with the current ones because of user handling. The buildpack spec requires that a fixed UID/GID be used, which is incompatible with how our current images mount NFS.