Page MenuHomePhabricator

Pipeline: provide a way to rebuild all blubber images
Open, LowPublic

Description

Currently the CD pipeline rebuilds production images from scratch, not using any layer caching in Docker so rebuilding all production images for issues in base layers should be as simple as re-triggering the pipeline for an image; however, there is no easy way to trigger a rebuild for all images produced by the pipeline.

Ideally there would be a mechanism by which someone could retrigger a build of all images similar to the way docker-pkg is used currently.

Event Timeline

The general idea (specified in some ticket I lost track of, to be added later) for long-term support is:

  • when an image is ready to be published (or is published), we register it with debmonitor (see debmonitor.wikimedia.org), which also collects feedback from clair, so that it can know of the presence of an image, and what's installed in it, at which version
  • Debmonitor will then tell us which images (both coming from docker-pkg and blubber) have vulnerable software versions, and also know a tree of dependencies so that it can re-trigger the build for all of them.

Of course nothing like this is implemented at the moment, but we should get to it before the end of the FY hopefully.

For the time being, you can't just trigger a rebuild of your blubber images - I guess they use a specific tag of their base images, and will need to use the updated one.

Meta comment: I have no strong opinion on this, but I note that this task was opened in 2019, and is still untriaged. It would be good to, at least, decide if this is urgent or not.

Meta comment: I have no strong opinion on this, but I note that this task was opened in 2019, and is still untriaged. It would be good to, at least, decide if this is urgent or not.

Setting as low prio since there is a current workaround. I think this task is not urgent but important. I was reminded of this today after seeing: T274262: Rebuild all blubber build docker images running on kubernetes.

Meta comment: I have no strong opinion on this, but I note that this task was opened in 2019, and is still untriaged. It would be good to, at least, decide if this is urgent or not.

Setting as low prio since there is a current workaround. I think this task is not urgent but important. I was reminded of this today after seeing: T274262: Rebuild all blubber build docker images running on kubernetes.

Sorry but what happened with T274262 is *not* a workaround: several people had to rebuild their images by explicitly bumping the version in their repositories; we need to have a better mechanism in general.