- - Provide FQDN of system.
- - If other than a hard drive issue, please depool the machine (and confirm that it’s been depooled) for us to work on it. If not, please provide time frame for us to take the machine down.
- - Put system into a failed state in Netbox.
- - Provide urgency of request, along with justification (redundancy, dependencies, etc): Medium to high as we already have 5 servers of this k8s cluster in this exact state
- - Describe issue and/or attach hardware failure log. (Refer to https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook if you need help)
- - Assign correct project tag and appropriate owner (based on above). Also, please ensure the service owners of the host(s) are added as subscribers to provide any additional input.
I am seeing the following in logs after a failed reimage of wikikube-worker1243.eqiad.wmnet
racadm>>getsel Record: 1 Date/Time: 07/15/2024 21:30:04 Source: system Severity: Ok Description: Log cleared. ------------------------------------------------------------------------------- Record: 2 Date/Time: 07/15/2024 21:38:46 Source: system Severity: Ok Description: OEM software event. ------------------------------------------------------------------------------- Record: 3 Date/Time: 07/15/2024 21:38:46 Source: system Severity: Ok Description: OEM software event. ------------------------------------------------------------------------------- Record: 4 Date/Time: 07/15/2024 21:38:49 Source: system Severity: Ok Description: OEM software event. ------------------------------------------------------------------------------- Record: 5 Date/Time: 07/15/2024 21:38:58 Source: system Severity: Ok Description: C: boot completed. ------------------------------------------------------------------------------- Record: 6 Date/Time: 07/15/2024 21:38:58 Source: system Severity: Ok Description: OEM software event. ------------------------------------------------------------------------------- Record: 7 Date/Time: 01/06/2025 09:52:02 Source: system Severity: Critical Description: The System Configuration Check operation resulted in the following issue: Comm Error: Backplane 0. -------------------------------------------------------------------------------