I've marked the PKI hosts in https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2024_Purge as "In Use" since deployment-prep and other projects rely on them to issue TLS certs.
The nodes should be upgraded to Bullseye or Bookworm, but I am not 100% sure what would be the best procedure to follow.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Remove CFSSL k8s-related auth_keys for cloud | operations/puppet | production | +0 -20 |
Moved pki-test01 to Bullseye, I didn't know that dist-upgrade.sh was present in the puppet repo so I've done it manually.
Important: Puppet seems broken currently on various nodes, and I noticed that the client token in /etc/cfssl/client-cfssl.conf is not the right one. I think that the project's puppet server was upgraded to Bookworm, and somehow the local commits containing values like profile::pki::client::auth_key are gone. So I added a local commit to the puppetserver's /srv/git/labs/private repo that changes hieradata/commons.yaml in this way:
profile::pki::client::auth_key: aaaabbbbccccdddd # PRIVATE DATA! This auth key comes from the PKI # cloud infrastructure, and it shouldn't go in # the "fake" private repo. # See T360595,T363829 -- elukey, 17/05/2024 profile::pki::client::auth_key: REDACTED
In this way we can override the fake-private-public-value aaaabbbbccccdddd with the real one. I have used the same technique for deployment-prep in T360595.
Two VMs left! pki-root and pki-intermediate have to connect to the mysql db instance on pki-db, but it seems that the related hiera settings (profile::pki::root_ca::db_pass and profile::pki::multirootca::db_pass) are not present in the puppet private repo anymore.
Tried to add some local config but failed, will restart on Monday.
Next steps:
Change #1034079 had a related patch set uploaded (by Elukey; author: Elukey):
[operations/puppet@production] Remove CFSSL k8s-related auth_keys for cloud
Change #1034079 merged by Elukey:
[operations/puppet@production] Remove CFSSL k8s-related auth_keys for cloud
@Andrew Hi! I have manually dist-upgraded all the VMs of the PKI project (except the puppetserver one) to Debian Bullseye, because re-creating the VMs would have been a little more painful. Would it be possible to fix their OS detail on your end to avoid them being listed to be on Buster (if it is possible of course).
Thanks a lot!