Possible frdb2004 hardware failure.
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Jgreen
	Apr 24 2025, 12:21 PM

Description

frdb2004 locked up this AM with multiple "CPU stuck" errors logged by the kernel.

On reboot, it is hanging with this message:

iDRAC Settings: 
HWC8010: The System Configuration Check operation resulted in the following
issue: Comm Error: Backplane 0.

Maybe reseating the backplane would help? Maybe it's fried?

The server is has been powered down.

Event Timeline

Jgreen created this task.Apr 24 2025, 12:21 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 24 2025, 12:21 PM

Jgreen triaged this task as Unbreak Now! priority.Apr 24 2025, 12:22 PM

Jgreen updated the task description. (Show Details)

Jgreen added a project: ops-codfw.Apr 29 2025, 2:05 PM

@Jhancock.wm when you have a minutes can you please check this host. Also can you also please upgrade the CPLD.

Thank you

Maintenance_bot added a project: SRE.Apr 29 2025, 2:29 PM

@Jgreen reseated all the connections to the backplane. server came up. I checked the firmware version of the CPLD and it is current (1.0.7).
lemme know if it all looks good to you.

Jhancock.wm moved this task from Backlog to Hardware Failure / Troubleshoot on the ops-codfw board.Apr 29 2025, 4:03 PM

In T392579#10777103, @Jhancock.wm wrote:

@Jgreen reseated all the connections to the backplane. server came up. I checked the firmware version of the CPLD and it is current (1.0.7).
lemme know if it all looks good to you.

Thanks @Jhancock.wm, looks good--it's back in mariadb replication and looks back to normal.

Possible frdb2004 hardware failure.Closed, ResolvedPublicActions

Description

Event Timeline

Possible frdb2004 hardware failure.
Closed, ResolvedPublic
Actions