Page MenuHomePhabricator

db1050 raid degraded
Closed, ResolvedPublic

Description

Slot Number: 5
Firmware state: Failed

Event Timeline

Springle raised the priority of this task from to Needs Triage.
Springle updated the task description. (Show Details)
Springle added subscribers: Springle, jcrespo.

Jaime and Sean,

db1050 ...well db1001-db1050 are all out of warranty. I've used up the last stock of 300GB SAS disks. Do you want me to order more or is it time to start planning replacement servers?

Chris

@Cmjohnson @Springle db1002-db1007 are not being used as far as I know. How crazy would be to decommission some or of all of them and use their disks to fix db1050 + having a pool of replacements?

From my end it's not difficult at all. Let's wait for @Springle to comment.

Also, yes, time to plan another batch.

Joe triaged this task as High priority.Jul 13 2015, 5:57 PM
Joe set Security to None.
fgiunchedi lowered the priority of this task from High to Medium.Jul 21 2015, 2:47 PM
fgiunchedi subscribed.

I didn't notice this before but db1049/1050 are 600GB SAS drives not 300GB like the earlier batches of db servers. I am not going to be able to pull a disk from one of the recent decom'd servers. I will request to purchase a disk

https://rt.wikimedia.org/Ticket/Display.html?id=9565 is purchase request.

We killed of es1005 and re-purposed one of the disks for db1050. The disk is in rebuild state

Enclosure Device ID: 32
Slot Number: 5
Drive's position: DiskGroup: 0, Span: 2, Arm: 1
Enclosure position: N/A
Device Id: 5
WWN: 5000C50054E92240
Sequence Number: 20
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 558.911 GB [0x45dd2fb0 Sectors]
Non Coerced Size: 558.411 GB [0x45cd2fb0 Sectors]
Coerced Size: 558.375 GB [0x45cc0000 Sectors]
Sector Size: 0
Firmware state: Rebuild
Device Firmware Level: ES65
Shield Counter: 0
Successful diagnostics completion on : N/A
SAS Address(0): 0x5000c50054e92241
SAS Address(1): 0x0
Connected Port Number: 0(path0)
Inquiry Data: SEAGATE ST3600057SS ES656SL4FTPS
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :42C (107.60 F)
PI Eligibility: No
Drive is formatted for PI information: No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 6.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No

Cmjohnson claimed this task.

RAID has been restored

cmjohnson@db1050:~$ sudo megacli -PDList -aALL |grep "Firmware state:"
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up