Page MenuHomePhabricator

Failed disk in ms-be2028
Closed, ResolvedPublic

Description

Hi,

/dev/sdg in ms-be2028 has failed (kernel log later in this ticket), could you replace it, please?
/dev/sdg is array G / logical drive 7:

=> ld 7 show
Smart Array P840 in Slot 3
   array G
      Logical Drive: 7
[...]
         Disk Name: /dev/sdg

and array G contains physical drive 1I:1:1:

=> show config
[...]
   array G (SATA, Unused Space: 0  MB)

      logicaldrive 7 (3.6 TB, RAID 0, OK)

      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SATA, 4000.7 GB, OK)

I've put the indicator on with

pd 1I:1:1 modify led=on

And I'll shortly mark the drive as failed.

Aug 15 07:55:14 ms-be2028 kernel: [3445893.540529] sd 0:1:0:6: [sdg] tag#56 FAIL
ED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Aug 15 07:55:14 ms-be2028 kernel: [3445893.540535] sd 0:1:0:6: [sdg] tag#56 Sens
e Key : Medium Error [current] 
Aug 15 07:55:14 ms-be2028 kernel: [3445893.540539] sd 0:1:0:6: [sdg] tag#56 Add.
 Sense: Unrecovered read error
Aug 15 07:55:14 ms-be2028 kernel: [3445893.540543] sd 0:1:0:6: [sdg] tag#56 CDB:
 Read(16) 88 00 00 00 00 01 6f 0a 6e 30 00 00 00 10 00 00
Aug 15 07:55:14 ms-be2028 kernel: [3445893.540546] blk_update_request: critical medium error, dev sdg, sector 6157921840
Aug 15 07:55:14 ms-be2028 kernel: [3445893.576449] XFS (sdg1): metadata I/O error: block 0x16f0a6630 ("xfs_trans_read_buf_map") error 61 numblks 16
Aug 15 07:55:14 ms-be2028 kernel: [3445893.623162] XFS (sdg1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -61.
Aug 15 08:30:28 ms-be2028 kernel: [3448007.563746] XFS (sdg1): Unmounting Filesystem

Related Objects

Event Timeline

Mentioned in SAL (#wikimedia-operations) [2022-08-15T10:03:45Z] <Emperor> pd 1I:1:1 modify disablepd forced on ms-be2028 T315213

This server is out of warranty and I don't have any disk onsite for replacement

Change 823178 had a related patch set uploaded (by MVernon; author: MVernon):

[operations/puppet@production] swift: ms-be2028 /dev/sdg1 has failed

https://gerrit.wikimedia.org/r/823178

Change 823178 merged by MVernon:

[operations/puppet@production] swift: ms-be2028 /dev/sdg1 has failed

https://gerrit.wikimedia.org/r/823178

Drive replaced from a decom server.