Page MenuHomePhabricator

Degraded RAID on ms-be1053
Closed, ResolvedPublic

Description

TASK AUTO-GENERATED by Nagios/Icinga RAID event handler

A degraded RAID (ssacli) was detected on host ms-be1053. An automatic snapshot of the current RAID status is attached below.

Please sync with the service owner to find the appropriate time window before actually replacing any failed hardware.

CRITICAL: Slot 0: Failed: 1I:1:2 - OK: 1I:1:1, 1I:1:3, 1I:1:4, 2I:2:1, 2I:2:2, 2I:2:3, 2I:2:4, 3I:3:1, 3I:3:2, 3I:3:3, 3I:3:4, 4I:5:1, 4I:5:2 - Controller: OK - Battery/Capacitor: OK

$ sudo /usr/local/lib/nagios/plugins/get-raid-status-ssacli

HPE Smart Array P816i-a SR Gen10 in Slot 0 (Embedded)

   Array A

      Logical Drive: 1
         Size: 447.10 GB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Disabled
         Disk Name: /dev/sda 
         Mount Points: /srv/swift-storage/sda4 297.2 GB Partition Number 4, /srv/swift-storage/sda3 93.1 GB Partition Number 3
         OS Status: LOCKED
         Boot Volume: Primary
         Drive Type: Data
         LD Acceleration Method: Smart Path


   Array B

      Logical Drive: 2
         Size: 447.10 GB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Disabled
         Disk Name: /dev/sdb 
         Mount Points: /srv/swift-storage/sdb4 297.2 GB Partition Number 4, /srv/swift-storage/sdb3 93.1 GB Partition Number 3
         OS Status: LOCKED
         Boot Volume: Secondary
         Drive Type: Data
         LD Acceleration Method: Smart Path


   Array C

      Logical Drive: 3
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdc 
         Mount Points: /srv/swift-storage/sdc1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array D

      Logical Drive: 4
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: Failed
         MultiDomain Status: OK
         Caching:  Enabled
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array E

      Logical Drive: 5
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sde 
         Mount Points: /srv/swift-storage/sde1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array F

      Logical Drive: 6
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdf 
         Mount Points: /srv/swift-storage/sdf1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array G

      Logical Drive: 7
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdg 
         Mount Points: /srv/swift-storage/sdg1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array H

      Logical Drive: 8
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdh 
         Mount Points: /srv/swift-storage/sdh1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array I

      Logical Drive: 9
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdi 
         Mount Points: /srv/swift-storage/sdi1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array J

      Logical Drive: 10
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdj 
         Mount Points: /srv/swift-storage/sdj1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array K

      Logical Drive: 11
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdk 
         Mount Points: /srv/swift-storage/sdk1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array L

      Logical Drive: 12
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdl 
         Mount Points: /srv/swift-storage/sdl1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array M

      Logical Drive: 13
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdm 
         Mount Points: /srv/swift-storage/sdm1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache


   Array N

      Logical Drive: 14
         Size: 3.64 TB
         Fault Tolerance: 0
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Caching:  Enabled
         Disk Name: /dev/sdn 
         Mount Points: /srv/swift-storage/sdn1 3.6 TB Partition Number 1
         OS Status: LOCKED
         Drive Type: Data
         LD Acceleration Method: Controller Cache

Event Timeline

sdd is indeed busted and host is under warranty, please replace @Cmjohnson / @Jclark-ctr , thank you!

Thank you, I will get a ticket in with HPE ASAP

The case was submitted with HPE,
Successfully Submitted Case Number: 5355909720

HPE is sending the part, they sent me an email requesting duplicate information that I missed. Taken care of and the part should be here tomorrow or Tuesday.

replaced the disk