| MoritzMuehlenhoff | |
| Sep 5 2023, 7:51 AM |
mw2448 was reported to be unreachable during a scap deploy. There's nothing in SEL and I can't connect over the serial console, the output is simply blank.
Can you please have a look? I've depooled the server to "inactive", so you can shut it down any time.
We had stability issues with this server before (https://phabricator.wikimedia.org/T334429), so maybe this needs further investigation by Dell support.
@MoritzMuehlenhoff
I got into the idrac gui and there's nothing of interest to report. There's no BIOS or CPU errors this time. I do see something in the system lifecycle logs about several configuration changes and then the NIC bounced and the server powered off. I can't see what the changes were or if they're relevant in any way.
2023-09-04 18:12:52 SYS1001 System is turning off. 2023-09-04 18:12:52 LOG007 The previous log entry was repeated 1 times. 2023-09-04 18:12:22 SYS1003 System CPU Resetting. 2023-09-02 03:46:38 CTL38 The Patrol Read operation completed for RAID Controller in SL 3. 2023-09-02 03:00:00 CTL37 A Patrol Read operation started for RAID Controller in SL 3. 2023-08-31 13:55:08 SYS336 An existing hash value is updated because some system configuration items are changed. 2023-08-31 13:29:40 CTL129 The boot media of the Controller RAID Controller in SL 3 is Disk.Virtual.0:RAID.SL.3-1. 2023-08-31 13:25:41 SYS1005 The server power action is initiated because the host device initiated a warm-reset operation. 2023-08-31 13:25:10 SYS1003 System CPU Resetting. 2023-08-31 13:25:07 NIC101 The Embedded NIC 1 Port 1 network link is started. 2023-08-31 13:25:03 NIC100 The Embedded NIC 1 Port 1 network link is down. 2023-08-31 13:25:03 LOG007 The previous log entry was repeated 4 times. 2023-08-30 14:55:18 SYS336 An existing hash value is updated because some system configuration items are changed.
So far I've upgraded the idrac and the bios, rebooted the system, and it's now pinging. console isn't blank now.
I've also generated a report to send to Dell but I don't see anything on it that's new. The last error was in May and it's what the ticket you mentioned is about. I can still open a case with Dell if you want.
Thanks, let's retry with latest firmware, if anything happens again we can still open a case with Dell.
Mentioned in SAL (#wikimedia-operations) [2023-09-05T15:49:39Z] <claime> Repooled mw2448.eqiad.wmnet - T345597