Degraded RAID on db1171
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ops-monitoring-bot
	Apr 26 2025, 10:39 AM

Description

TASK AUTO-GENERATED by Nagios/Icinga RAID event handler

A degraded RAID (megacli) was detected on host db1171. An automatic snapshot of the current RAID status is attached below.

Please sync with the service owner to find the appropriate time window before actually replacing any failed hardware.

CRITICAL: 1 failed LD(s) (Degraded)

$ sudo /usr/local/lib/nagios/plugins/get-raid-status-megacli
=== RaidStatus (does not include components in optimal state)
name: Adapter #0

	Virtual Drive: 0 (Target Id: 0)
	RAID Level: Primary-1, Secondary-0, RAID Level Qualifier-0
	State: =====> Degraded <=====
	Number Of Drives: 10
	Number of Spans: 1
	Current Cache Policy: WriteBack, ReadAhead, Direct, No Write Cache if Bad BBU

		Span: 0 - Number of PDs: 10

			PD: 8 Information
			ERROR: =====> MISSING DRIVE INFO <=====

=== RaidStatus completed

Event Timeline

ops-monitoring-bot created this task.Apr 26 2025, 10:39 AM

Restricted Application added projects: DC-Ops, DBA. · View Herald TranscriptApr 26 2025, 10:39 AM

This is a backup source, do we have some spares available?

• Marostegui edited projects, added Data-Persistence; removed DBA.Apr 26 2025, 2:17 PM

Restricted Application added a project: DBA. · View Herald TranscriptApr 26 2025, 2:17 PM

• Marostegui removed a project: DBA.Apr 26 2025, 2:17 PM

Restricted Application added a project: DBA. · View Herald TranscriptApr 26 2025, 2:17 PM

Papaul moved this task from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.Apr 28 2025, 3:44 AM

• Marostegui moved this task from Triage to In progress on the DBA board.Apr 28 2025, 7:21 AM

wiki_willy added subscribers: VRiley-WMF, Jclark-ctr.Apr 28 2025, 5:58 PM

@VRiley-WMF & @Jclark-ctr - can you grab a spare from one of the decom'd servers for this?

In T392751#10770238, @Marostegui wrote:

This is a backup source, do we have some spares available?

VRiley-WMF claimed this task.Apr 30 2025, 7:16 AM

Swapped out the drive. Checked in with @Marostegui everything seems to be good. Closing this out.

Maintenance_bot moved this task from In progress to Done on the DBA board.Apr 30 2025, 8:29 AM

jcrespo subscribed.May 5 2025, 1:41 PM

Thank you, @Marostegui for taking care about this.

Degraded RAID on db1171Closed, ResolvedPublicActions

Description

Event Timeline

Degraded RAID on db1171
Closed, ResolvedPublic
Actions