Page MenuHomePhabricator

Report image metadata to debmonitor
Closed, ResolvedPublic

Description

Ideally, the process should be as follows:

  1. We scan the registry for images
  2. We check which of those images are currently in use directly in kubernetes, or indirectly as base for other images
  3. We run the docker report job on each of those images
  4. We clean up any image we downloaded.

IMHO this can be done in progressive steps, for instance item number 2 is not necerssary for an MVP, which can be limited to the other points. I'll start by building a lean way to submit said reports.

This can run on boron for now, but should really be moved to a dedicated VM in the future.

Details

Related Gerrit Patches:
operations/puppet : productionprofile::docker::reporter: periodically generate reports to debmonitor.
operations/docker-images/docker-report : masterInitial debianization
operations/docker-images/docker-report : masterAdd class to scan a registry for images
operations/docker-images/docker-report : masterFirst version of the debmonitor client

Event Timeline

Joe created this task.Dec 20 2019, 6:56 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 20 2019, 6:56 AM
Joe claimed this task.Dec 20 2019, 6:56 AM
Joe triaged this task as High priority.

Change 559165 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/docker-images/docker-report@master] First version of the debmonitor client

https://gerrit.wikimedia.org/r/559165

Change 559804 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/docker-images/docker-report@master] Add class to scan a registry for images

https://gerrit.wikimedia.org/r/559804

Change 559165 merged by Giuseppe Lavagetto:
[operations/docker-images/docker-report@master] First version of the debmonitor client

https://gerrit.wikimedia.org/r/559165

Mentioned in SAL (#wikimedia-operations) [2019-12-24T12:20:20Z] <volans@deploy1001> Started deploy [debmonitor/deploy@39ad186]: Release v0.2.2 - T241206

Mentioned in SAL (#wikimedia-operations) [2019-12-24T12:21:00Z] <volans@deploy1001> Finished deploy [debmonitor/deploy@39ad186]: Release v0.2.2 - T241206 (duration: 00m 40s)

Volans added a subscriber: Volans.Dec 24 2019, 12:23 PM

The issue for the DELETE has been fixed, I've successfully deleted the image docker-registry.wikimedia.org/python3-build-stretch:0.0.2 that was failing during the tests.
Please ensure that also the /upload endpoint still works as expected too.

Change 559804 merged by jenkins-bot:
[operations/docker-images/docker-report@master] Add class to scan a registry for images

https://gerrit.wikimedia.org/r/559804

Change 562503 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/docker-images/docker-report@master] Initial debianization

https://gerrit.wikimedia.org/r/562503

Change 562503 merged by Giuseppe Lavagetto:
[operations/docker-images/docker-report@master] Initial debianization

https://gerrit.wikimedia.org/r/562503

Mentioned in SAL (#wikimedia-operations) [2020-01-07T14:39:04Z] <_joe_> uploading python3-docker-report to {buster,stretch}-wikimedia, T241206

Change 562526 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] profile::docker::reporter: periodically generate reports to debmonitor.

https://gerrit.wikimedia.org/r/562526

Change 562526 merged by Giuseppe Lavagetto:
[operations/puppet@production] profile::docker::reporter: periodically generate reports to debmonitor.

https://gerrit.wikimedia.org/r/562526

Joe closed this task as Resolved.Jan 13 2020, 1:27 PM