Page MenuHomePhabricator

Make security updates of docker images manageable
Closed, DuplicatePublic

Description

The problem

When using docker images in production (like we plan to do with kubernetes), we want to be able to perform upgrades ideally as easy as we do for production machines.
Using a layered container system like docker makes this task much more complex that it is for physical machines or VMs: a software

The typical workflow when a security upgrade of a software/library is issued should be something like what follows:

  1. We find out which docker images on our registry contain said software. We rebuild and push new versions to the docker registry. A versioning convention needs to be established for both images and security updates.
  2. We find all the images that depend from those images and need to be rebuilt. We rebuild and push new versions to the docker registry
  3. For all of those images that are being run in kubernetes, do a new deployment to bring the newer image to production. This should be handled using the deployment tool we chose to use on kubernetes.

All of this can be wrapped into the same tool we use for general upgrades.

A possible solution

The software that manages upgrades needs to be able to query some database to get the list of images (including tags!) containing vulnerable versions of the software, and then for all images that are directly or indirectly based on it.

In order to obtain this, we could create a webservice with an API that this software can query, and that can store the relevant information on the docker images.

The API endpoints could be something like:

GET /packages/<package-name> returns the list of package versions found, and the associated list of image_name:tag that include that specific version. Not sure if we want to extend this to libraries we import via pip/npm
GET /images/<image_name:tag>/children returns a list of image_name:tag that use that specific image as a base.
GET /images/<image_name:tag/packages returns a list of packages direclty included in the image
POST /images/<image_name:tag> posts information about a new image to the service. The information included should at least include the base image name:tag and the list of packages installed at that stage and their respective versions.
PUT /images/<image_name:tag> modifies information about a preexisting image
DELETE /images/<image_name:tag> removes an image from the database.

We should then make sure that all of our images are uploaded to the registry with a process that registers the needed information to this service.

Event Timeline

T167504 has a proposal as well that at least partially (if not fully) overlaps. I think we should merge this one into T167504

FWIW the general approach looks ok I am a bit worried about

We should then make sure that all of our images are uploaded to the registry with a process that registers the needed information to this service.

This is the big elephant in the room and it's hiding in a single sentence. We should make sure only tools that do the above can push to the registry