Degraded RAID on db1129
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ops-monitoring-bot
	Jun 28 2021, 8:17 PM

Description

TASK AUTO-GENERATED by Nagios/Icinga RAID event handler

A degraded RAID (megacli) was detected on host db1129. An automatic snapshot of the current RAID status is attached below.

Please sync with the service owner to find the appropriate time window before actually replacing any failed hardware.

CRITICAL: 1 failed LD(s) (Degraded)

$ sudo /usr/local/lib/nagios/plugins/get-raid-status-megacli
=== RaidStatus (does not include components in optimal state)
name: Adapter #0

	Virtual Drive: 0 (Target Id: 0)
	RAID Level: Primary-1, Secondary-0, RAID Level Qualifier-0
	State: =====> Degraded <=====
	Number Of Drives: 6
	Number of Spans: 1
	Current Cache Policy: WriteBack, ReadAhead, Direct, No Write Cache if Bad BBU

		Span: 0 - Number of PDs: 6

			PD: 3 Information
			ERROR: =====> MISSING DRIVE INFO <=====

=== RaidStatus completed

Related Objects

Mentioned In: T287481: Degraded RAID on db1129

Event Timeline

ops-monitoring-bot created this task.Jun 28 2021, 8:17 PM

Restricted Application added a subscriber: • Marostegui. · View Herald TranscriptJun 28 2021, 8:17 PM

Can we get a rma for this failed disk?

Thanks

wiki_willy assigned this task to • Cmjohnson.Jun 28 2021, 8:35 PM

Hi @Cmjohnson - just a heads up, there's only a couple more months before the warranty expires on this host. Thanks, Willy

• Marostegui moved this task from Triage to In progress on the DBA board.Jun 30 2021, 8:06 AM

Ticket opened with Dell

• Cmjohnson moved this task from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.Jul 1 2021, 3:56 PM

@Jclark-ctr did this disk arrive?

wiki_willy reassigned this task from • Cmjohnson to Jclark-ctr.Jul 21 2021, 4:40 PM

wiki_willy added a subscriber: • Cmjohnson.

• Marostegui mentioned this in T287481: Degraded RAID on db1129.Jul 27 2021, 2:58 PM

received replacement drive

@Marostegui can this drive be replaced?

Yes please

disk has been replaced @Marostegui

Jclark-ctr closed this task as Resolved.Jul 28 2021, 8:21 PM

Thanks John!
All looking good:

root@db1129:~# megacli -LDInfo -Lall -aALL


Adapter 0 -- Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 4.364 TB
Sector Size         : 512
Is VD emulated      : Yes
Mirror Data         : 4.364 TB
State               : Optimal

Degraded RAID on db1129Closed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Degraded RAID on db1129
Closed, ResolvedPublic
Actions