Sorry for late replies. I have extremely limited time. I will try to get back in (I've had issues) and resolve over next 2-3 weeks, starting next Friday.

Hi @Beetstra! Did you ever get your access issues sorted out? I need to rebuild the host that these VMs are on soon (see T364457 for probably-uninteresting context).

Hello, I would like the VM etytree-a to be kept as a copy, as I am planning an upgrade of the operating system, but I need the old copy as a reference to build the updated one. The issue is that I use Openlink Virtuoso as a database and I need time to figure out how to manage the upgrade. @fnegri kindly helped me but I did not find the time to proceed. I think by the end of October I will have sorted the issue out and I will create and update version of the etytree application. Thanks!

In T367536#10076979, @Epantaleo wrote:

Hello, I would like the VM etytree-a to be kept as a copy, as I am planning an upgrade of the operating system, but I need the old copy as a reference to build the updated one. The issue is that I use Openlink Virtuoso as a database and I need time to figure out how to manage the upgrade. @fnegri kindly helped me but I did not find the time to proceed. I think by the end of October I will have sorted the issue out and I will create and update version of the etytree application. Thanks!

I linked Ester to the wrong ticket; I've copied this comment to T367529

@Beetstra do you still plan to work on the upgrade?

The VMs are now also alerting with the following messages:

• Puppet CA certificate coibot.linkwatcher.eqiad.wmflabs is about to expire in 1d 10h 56m 37s
• Puppet CA certificate liwa3.linkwatcher.eqiad.wmflabs is about to expire in 1d 11h 0m 31s

This will probably cause more errors starting tomorrow when the certs expire. I can check if it's possible to renew these certs, but we will still need to shut down those Buster VMs and replace them with Bookworm VMs, ideally before the end of September.

Mentioned in SAL (#wikimedia-cloud-feed) [2024-09-06T13:51:30Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.vps.refresh_puppet_certs on coibot.linkwatcher.eqiad1.wikimedia.cloud (T367536)

Mentioned in SAL (#wikimedia-cloud-feed) [2024-09-06T13:51:53Z] <fnegri@cloudcumin1001> END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on coibot.linkwatcher.eqiad1.wikimedia.cloud (T367536)

Mentioned in SAL (#wikimedia-cloud-feed) [2024-09-06T14:07:52Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.vps.refresh_puppet_certs on coibot.linkwatcher.eqiad1.wikimedia.cloud (T367536)

Mentioned in SAL (#wikimedia-cloud-feed) [2024-09-06T14:08:15Z] <fnegri@cloudcumin1001> END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on coibot.linkwatcher.eqiad1.wikimedia.cloud (T367536)

Mentioned in SAL (#wikimedia-cloud-feed) [2024-09-06T14:19:00Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.vps.refresh_puppet_certs on liwa3.linkwatcher.eqiad1.wikimedia.cloud (T367536)

Mentioned in SAL (#wikimedia-cloud-feed) [2024-09-06T14:19:27Z] <fnegri@cloudcumin1001> END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on liwa3.linkwatcher.eqiad1.wikimedia.cloud (T367536)

The cookbook failed but I renewed the certs manually, the alerts are no longer firing.

I have moved the bots to two new instances, using bookworm, and re-mounted volumes for their data. Old volumes and old instances are deleted. Bots are all three running fine.