Alert in need of triage: KubernetesWorkerUnschedulable
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	tappof
	Aug 1 2025, 7:09 AM

Description

The alert KubernetesWorkerUnschedulable has started firing 1 month ago.

alertname=KubernetesWorkerUnschedulable
node=wikikube-worker2100.codfw.wmnet
prometheus=k8s
severity=warning
site=codfw
source=prometheus
team=sre

Name	Content
dashboard	https://grafana.wikimedia.org/d/G8zPL7-Wz/kubernetes-node?orgId=1&var-dc=codfw%20prometheus%2Fk8s&var-instance=wikikube-worker2100.codfw.wmnet
description	Kubernetes worker wikikube-worker2100.codfw.wmnet has been unschedulable for >= 24h k8s@codfw
runbook	https://wikitech.wikimedia.org/wiki/Kubernetes
summary	Kubernetes worker wikikube-worker2100.codfw.wmnet has been unschedulable for >= 24h k8s@codfw

Triage metadata. Do not delete.

fingerprint=4d20cd61d838f7dd

Mentioned Here: T409103: decommission wikikube-worker[2052-2054,2063,2079-2084,2096-2101].codfw.wmnet
T409104: decommission wikikube-worker[2116-2123,2216-2241].codfw.wmnet

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 1 2025, 7:09 AM

This host was set aside for mw-experimental work by @jijiki, I'll silence the alert for a month.

sorry folks, host's number is up for retirement, my bad. tx @Clement_Goubert

jijiki triaged this task as Low priority.Aug 1 2025, 11:01 AM

Just a heads up that the alert fired again, can it be silenced for another month?

Silenced for 30 days again

Just a heads up that the alert fired again, can it be silenced for another month?

Done

And one more time :)

Silencing for 1 month, decoms are in progress.

@jasmine_ please resolve this task when done with T409104 T409103

In T400969#11529307, @Clement_Goubert wrote:

@jasmine_ please resolve this task when done with T409104 T409103

Done, thanks all~