Page MenuHomePhabricator

toolforge: Move k8s etcd nodes to larger instance flavors
Closed, ResolvedPublic

Description

While investigating T333922: toolforge k8s control plane freezing and other stability issues we noticed that the Toolforge Kubernetes etcd nodes are undersized for the task as they're using the g3.cores1.ram2.disk20.localdisk flavor. We need to migrate those instances to a larger flavor.

Event Timeline

aborrero triaged this task as Medium priority.Apr 4 2023, 11:56 AM
aborrero moved this task from Inbox to Soon! on the cloud-services-team board.
aborrero moved this task from Triage to Backlog on the Toolforge board.

Cloudvirtlocal1002 and Cloudvirtlocal1003 are now in service and ready for new etcd nodes. These new nodes should replace tools-k8s-etcd-14, tools-k8s-etcd-15, toolsbeta-test-k8s-etcd-18, toolsbeta-test-k8s-etcd-19.

Cloudvirtlocal1001 is still out of service pending some datacenter work. When that's done we can replace tools-k8s-etcd-13 and toolsbeta-test-k8s-etcd-17 with new nodes running on cloudvirtlocal1001.