That would allow to reduce considerably the amount of jobs harbor has to run and keep track of.
This would be specially useful if we remove all the empty projects (as we create them on demand).
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | dcaro | T342563 [harbor] Stabilize the current installation | |||
| Resolved | dcaro | T344433 [harbor] See if we can replace the per-project cleanup policy with a harbor-wide one |
Event Timeline
Comment Actions
A difference between the two systems:
- The global garbage collection will cleanup any non-referenced objects, it does not have the knowledge of "the last 5 pushed ones" or similar, so it's way simpler.
- The per-project one is quite flexible, allowing to filter by tag, or the last N pushed/pulled objects
Comment Actions
Ok, so as long as all we want to cleanup is untagged images, the global GC can do.
This is, when the build service pushes a new image, it always does it with the 'latest' tag, so the old one gets untagged. Then on the next GC run it will be cleanup up.
The problem comes when the user has exceeded the quota, then the user will get errors when creating new images until the GC runs and cleans up the old ones :/
The ideal (according to me xd) would be that the images are cleanup up only when needed, this is, before pushing a new one.
In any case, let me run the GC on tools, see how much it takes to run, if it's not too heavy we can replace the policies with it as we don't really use the extra flexibility yet as users are only allowed one tag.
Comment Actions
Ok, so currently it takes ~30s to run, but it's not really removing anything as the per-project ones run every 5min xd
I propose to play with it after we remove all the empty projects T344435: [harbor] Cleanup empty projects, as that will for sure be less heavy than trying to run on +3k projects.
Comment Actions
With the reduction of projects and on-demand creation, this is not relevant anymore, will reconsider when we start scaling up.