For testing and building out the CD pipeline in toolsbeta, we need to have a separate docker-registry and image builder rather than relying on the main Toolforge one.
Description
Description
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T194332 [Epic] Make Toolforge a proper platform as a service with push-to-deploy and build packs | |||
| Resolved | Bstorm | T265684 Figure out CI solution for building buildpack-based images for Toolforge | |||
| Open | Bstorm | T267374 Set up a Toolforge buildpack CI pipeline as a POC | |||
| Open | Bstorm | T267616 Set up docker-registry and image builder infra in toolsbeta | |||
| Resolved | Andrew | T267618 Request increased quota for toolsbeta Cloud VPS project |
Event Timeline
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2020-11-10T18:27:06Z] <legoktm> creating toolsbeta-docker-imagebuilder-01 (T267616)
Comment Actions
Change 707572 had a related patch set uploaded (by Bstorm; author: Bstorm):
[operations/puppet@production] toolforge harbor: puppetize experimental base server for harbor
Comment Actions
Change 707572 merged by Bstorm:
[operations/puppet@production] toolforge harbor: puppetize experimental base server for harbor
Comment Actions
Ok, so I have a very nice https://goharbor.io server running in toolsbeta (using independent database auth for testing) https://harbor.toolsbeta.wmflabs.org
It has APIs for adding just about everything and has robot auth, etc. It grant plenty of flexibility for image management (even quotas).
It hooks up to LDAP, but that's not necessarily helpful here. I think to do this right, users need to not have direct push rights. That needs to be handled by another build service. However, with this, that service can easily run in k8s.
Comment Actions
In order to deploy this to tools soon, I'm proposing replacing tools-docker-registry with harbor running in k8s. However, unless we deploy cinder for k8s, we'll be doing that with NFS storage.
Comment Actions
I've deployed harbor in toolsbeta, however, I want to modify the deployment a bit before replacing docker-registry with it or putting it in tools.
Comment Actions
Change 722664 had a related patch set uploaded (by Bstorm; author: Bstorm):
[operations/puppet@production] toolforge harbor: install clients for redis and postgres
Comment Actions
Change 722664 merged by Bstorm:
[operations/puppet@production] toolforge harbor: install clients for redis and postgres