The validating admission webhooks running in the new version of Toolforge Kubernetes use certs generated using the certificates API in Kubernetes. That implies an expiration of one year. They need to be renewed via an automated or manually reviewed process that can be reliably used without a deep dive into that API every time.
|Open||• bd808||T232536 Toolforge Kubernetes internal API down, causing `webservice` and other tooling to fail|
|Open||None||T236565 "tools" Cloud VPS project jessie deprecation|
|Open||None||T214513 Upgrade Toolforge Kubernetes|
|Stalled||Bstorm||T215553 Figure out cert management for Toolforge kubernetes and make it clear in documents, etc. for the upgrade|
|Resolved||Bstorm||T215678 Replace each of the custom controllers with something in a new Toolforge Kubernetes setup|
|Open||Bstorm||T238162 Establish a process for renewing TLS certs for the 2 webhook controllers|
@aborrero set up a script for requesting certs at modules/toolforge/files/k8s/admin_scripts/wmcs-k8s-get-cert.sh that places certs in a tmpdir. The main difference here is that we need them in a Kubernetes secret (which can easily be generated from the files like in their individual scripts) and to restart the services.
The services also could use some prometheus instrumentation to tell our monitors about when their certs expire.