Maniphest T412078

Alert in need of triage: SmartNotHealthy (instance sretest2006:9100)
Open, MediumPublic
Actions

Assigned To

Authored By

	LSobanski
	Dec 9 2025, 7:40 AM

Description

The alert SmartNotHealthy has started firing 1 month ago.

Labels

alertname=SmartNotHealthy
cluster=misc
device=sda
instance=sretest2006:9100
job=node
prometheus=ops
severity=warning
site=codfw
source=prometheus
team=sre

Annotations

Name	Content
dashboard	https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=sretest2006
description	The disk SMART status is not healthy, this could be an early warning before the disk fails.
runbook	https://wikitech.wikimedia.org/wiki/SMART#Alerts
summary	Disk not healthy

Links

Triage metadata. Do not delete.

fingerprint=fe1655842b4b1212

Event Timeline

LSobanski created this task.Dec 9 2025, 7:40 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 9 2025, 7:40 AM

LSobanski added a project: ops-codfw.Dec 15 2025, 3:38 PM

Jhancock.wm moved this task from Backlog to Hardware Failure / Troubleshoot on the ops-codfw board.Dec 15 2025, 4:19 PM

Maintenance_bot added a project: SRE.Dec 15 2025, 4:29 PM

if you zoom out to half a year, this alert has been active since the end of July. Could this have been a mistake on initial setup? Would a reimage fix it? it is an OS drive, not a storage drive. I am not sure what it's being used for at this moment.

LSobanski triaged this task as Low priority.Jan 5 2026, 3:24 PM

MoritzMuehlenhoff claimed this task.Jan 5 2026, 3:24 PM

MoritzMuehlenhoff raised the priority of this task from Low to Medium.

Jhancock.wm moved this task from Hardware Failure / Troubleshoot to Blocked on the ops-codfw board.Fri, Apr 10, 4:29 PM

Alert in need of triage: SmartNotHealthy (instance sretest2006:9100)Open, MediumPublicActions