elastic2018 was rebooted during normal maintenance, but never came back up.
console shows that it is stuck in the boot process, trying to boot over PXE and failing.
@Volans had a look at logs and found:
date=05/30/2018 time=23:32 description=POST Error: 266-Non-Volatile Memory Corruption Detected. Configuration settings restored to defaults. If enabled, Secure Boot security settings may be lost. Action: Restore desired configuration settings. Contact HP if issue persists.
Mentioned in SAL (#wikimedia-operations) [2018-05-31T08:42:55Z] <gehel> power off elastic2018 - T196045
Having a look around in the system utility (ESC+9) I found that:
System Health Summary > System BIOS > Health Status: Configuration Required
So it might be that somehow the BIOS config was lost during the reboot (or before?).
I had a quick look but I'm not fully sure of what are all the correct settings.
I'd recommend to have DC-Ops have a thorough check of all the BIOS options to make sure they are correct (not only boot order at this point, but all the other settings that we set on new installations too, cannot trust it at this point).
@Papaul could you have a look at elastic2018 and see if you understand anything? The server is powered off, do anything you'd like with it...
@Gehel for some reason, the server lost some settings like in the BIOS Serial Console & EMS EMS Console was COM1 , BOOT options was set to UEFI and USB options Internal SD Card Slot was enable. I changed back those settings the way it supposed to be. Power off and on the server 3 times using the ILO CLI . Server is back up
It looks like this worked, elastic2018 looks good again.
@Papaul is there any follow up we should do on that? Otherwise, feel free to close the task. Thanks!