Looking at something completely different I noticed that the Debmonitor garbage collection is not running since a while. Upon checking the logs the issue is due to:
django.db.models.deletion.ProtectedError: ("Cannot delete some instances of model 'PackageVersion' because they are referenced through a protected foreign key: 'ImagePackage.upgradable_imageversion'"
This happens because when we added the image support to the GC, it was added just for the packages referenced by images and not also for the upgradable packages in the images.
Unfortunately the fix is not that trivial because adding an additional level of join multiplies the number of rows to traverse to sizes that are not anymore feasible. So it needs to be refactored to do smaller queries and a smarter deletion.
Why we didn't notice?
Because the GC runs in a crontab and uses systemd-cat to log to syslog, not generating any output on error, hence not sending a cron-spam email. This bit too should be fixed to make sure we get notified in case of the GC not running.