When buildkitd is asked by a client to push or pull from a registry, it asks the client in turn for registry credentials. The credential may be a passthrough token or username/password pair that are used to acquire a token. Either way, the resulting token is then cached in memory (a go map) by buildkitd's auth handler using only the scope (image ref and registry action, e.g. blubber:sometag,push) and no client specific or session information. This means that registry credentials leak between client connections.

I've reported this to upstream.

i've run a test with two clients: client A has a ~/.docker/config.json set up with an auths section and a valid token for a given registry; client B has no ~/.docker/config.json at all. client A builds an image and publishes it to the registry, having its auth token retrieved successfully and used by the resolver. client B builds its own image and pushes it with the same image ref just seconds after client A. buildkitd uses the cached auth token from client A and successfully pushes the image from client B to the registry

Upstream does not strictly consider this a security issue and have stated that for the moment we should assume all clients to be mutually trusting. (I'm not embargoing this discussion for that reason.) They do seem, however, open to features that would change this behavior.

For us and our current use case, this seems like a potential vector and needs to be patched before we depend at all on buildkitd from GitLab trusted runners.

As a first step, I'd like to:

• fork buildkitd and maintain our own mirror repo in GitLab (https://gitlab.wikimedia.org/repos/releng/buildkit)
• branch from upstream v0.10 (wmf/v0.10)
• patch the image resolver (part of code responsible for pushing and pulling images) to key the auth tokens using the session group's IDs, isolating the cache to each session
• build and publish`buildkitd` image to WMF registry from fork (published as docker-registry.wikimedia.org/repos/releng/buildkit:wmf-v0.10-1)
• edit buildkitd image ref in puppet and restart buildkitd service on gitlab-runner hosts