As of today, the wmcs.toolforge.k8s.reboot cookbook will force-reboot only if the normal drain/reboot/uncordon procedure fails.
In some circumstances, for example when we know all nodes are stuck, we want to just do a rolling force-reboot of the fleet, without waiting for the timeouts.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | None | T333922 [k8s,infra] k8s control plane freezing and other stability issues | |||
| Resolved | • aborrero | T333379 toolforge kubernetes: create roll-reboot cookbook | |||
| Open | None | T356969 toolforge k8s automation: introduce option to force reboot each node directly |