Page MenuHomePhabricator

Disk (sdl) failed in ms-be1068
Closed, ResolvedPublic

Description

Hi,

sdl has failed in ms-be1068; could it be swapped out ASAP, please? You can work on this system at any time.

lshw -C disk says bus info: scsi@0:2.12.0
megacli -ldpdinfo -a0 tells us Target Id: 12 is associated with
Enclosure Device ID: 32, Slot Number: 10
[and has Media Error Count: 83]
I have hopefully enabled the locator light with
megacli -PDLocate -PhysDrv [32:10] -a0

Kernel log extract:

Jan 27 19:09:24 ms-be1068 kernel: [12811023.641583] megaraid_sas 0000:18:00.0: 3288 (759697588s/0x0001/FATAL) - Uncorrectable medium error logged for VD 0c/a at 39c9d4b
0 (on PD 0a(e0x20/s10) at 39c9d4b0)
Jan 27 19:09:24 ms-be1068 kernel: [12811023.647315] sd 0:2:12:0: [sdl] tag#565 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.660636] sd 0:2:12:0: [sdl] tag#550 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.661502] sd 0:2:12:0: [sdl] tag#549 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.669049] sd 0:2:12:0: [sdl] tag#549 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.671673] sd 0:2:12:0: [sdl] tag#550 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.713836] sd 0:2:12:0: [sdl] tag#532 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.716276] sd 0:2:12:0: [sdl] tag#536 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.724692] sd 0:2:12:0: [sdl] tag#28 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.726890] sd 0:2:12:0: [sdl] tag#29 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.726920] sd 0:2:12:0: [sdl] tag#29 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=13s
Jan 27 19:09:24 ms-be1068 kernel: [12811023.726928] sd 0:2:12:0: [sdl] tag#29 Sense Key : Medium Error [current] 
Jan 27 19:09:24 ms-be1068 kernel: [12811023.726932] sd 0:2:12:0: [sdl] tag#29 Add. Sense: No additional sense information
Jan 27 19:09:24 ms-be1068 kernel: [12811023.726937] sd 0:2:12:0: [sdl] tag#29 CDB: Read(16) 88 00 00 00 00 00 39 c9 d4 a0 00 00 00 20 00 00
Jan 27 19:09:24 ms-be1068 kernel: [12811023.726944] blk_update_request: I/O error, dev sdl, sector 969528480 op 0x0:(READ) flags 0x1000 phys_seg 4 prio class 0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.732002] sd 0:2:12:0: [sdl] tag#30 BRCM Debug mfi stat 0x2d, data len requested/completed 0x4000/0x0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.738017] sd 0:2:12:0: [sdl] tag#30 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=2s
Jan 27 19:09:24 ms-be1068 kernel: [12811023.738027] sd 0:2:12:0: [sdl] tag#30 Sense Key : Medium Error [current] 
Jan 27 19:09:24 ms-be1068 kernel: [12811023.738031] sd 0:2:12:0: [sdl] tag#30 Add. Sense: No additional sense information
Jan 27 19:09:24 ms-be1068 kernel: [12811023.738036] sd 0:2:12:0: [sdl] tag#30 CDB: Read(16) 88 00 00 00 00 00 05 06 3e e0 00 00 00 20 00 00
Jan 27 19:09:24 ms-be1068 kernel: [12811023.738041] blk_update_request: I/O error, dev sdl, sector 84295392 op 0x0:(READ) flags 0x1000 phys_seg 4 prio class 0
Jan 27 19:09:24 ms-be1068 kernel: [12811023.738086] XFS: metadata IO error: 9 callbacks suppressed
Jan 27 19:09:24 ms-be1068 kernel: [12811023.738205] XFS (sdl1): metadata I/O error in "xfs_imap_to_bp+0x61/0xb0 [xfs]" at daddr 0x39c9cca0 len 32 error 5
Jan 27 19:09:24 ms-be1068 kernel: [12811023.749200] XFS (sdl1): metadata I/O error in "xfs_imap_to_bp+0x61/0xb0 [xfs]" at daddr 0x50636e0 len 32 error 5
Jan 27 19:09:24 ms-be1068 kernel: [12811023.759769] XFS (sdl1): xfs_do_force_shutdown(0x1) called from line 296 of file fs/xfs/xfs_trans_buf.c. Return address = 000000002e106861
Jan 27 19:09:24 ms-be1068 kernel: [12811023.770103] XFS (sdl1): I/O Error Detected. Shutting down filesystem
Jan 27 19:09:24 ms-be1068 kernel: [12811023.776737] XFS (sdl1): Please unmount the filesystem and rectify the problem(s)

Looking back through kernel logs, this drive has been resulting in errors since at least 2024-01-01.

Event Timeline

MatthewVernon created this task.

Started case with dell ordered replacement drive.
You have successfully submitted request SR184210022.

In mean time i have swapped 8tb failed drive with one we have available and will add rma drive to our onhands when it arrives @MatthewVernon

Jclark-ctr lowered the priority of this task from High to Low.

@Jclark-ctr thank you for the quick swap, much appreciated :-)

received new drive and returned failed drive