Page MenuHomePhabricator

Clean-up / delete old versions of service pipeline created docker images from the public docker registry?
Open, Needs TriagePublic

Description

The release pipeline is brilliant, but the docker registry ends up with dozens or even hundreds of tags of images over years which are unnecessary and confusing. Can we prune them out of the way (or just hide them) so that we don't end up with e.g. https://docker-registry.wikimedia.org/wikimedia/mediawiki-services-function-orchestrator/tags/ with > 2x200 old image versions no-one is ever going to look at?

Event Timeline

The Toolhub and Wikimedia-Developer-Portal projects are both also publishing an image for each merged commit and will eventually make someone sad if we don't add pruning for older builds.

I think this is a duplicate of T242604. Unless someone objects, I 'll merge it in that one in a couple of days.

I 'll also add it to that task but hiding wise, it should be easy to do so, code is at https://gerrit.wikimedia.org/g/operations/puppet/+/refs/heads/production/modules/docker_registry_ha/files/registry-homepage-builder.py, we could set a limit on say the last 10 images. We can also handle it in Javascript and hide part of the list under some collapsable list/box.

I think this is a duplicate of T242604. Unless someone objects, I 'll merge it in that one in a couple of days.

For this one in particular I was thinking that maybe blubber would have special rights on pushing new image N to also delete the N~10 image from the registry or whatever, but if that's not the way we're going to do it then just merging seems fine.

I think this is a duplicate of T242604. Unless someone objects, I 'll merge it in that one in a couple of days.

For this one in particular I was thinking that maybe blubber would have special rights on pushing new image N to also delete the N~10 image from the registry or whatever, but if that's not the way we're going to do it then just merging seems fine.

Blubber doesn't have any kind of special rights, it's just a Blubber YAML-> Dockerfile translation layer. The Pipeline (Jenkins specifically, executing docker commands currently) talks to the registry, but it just tags the new image and pushes it. It doesn't have any knowledge of prior images, aside from what may be around in the local docker instance - which is however pruned regularly due to space constraints.