Hello Papaul,
As described in T281621 , elastic2033 can't boot. It gets to a PXE booting screen, but no further. I've done a 'power off hard/power on' from the iLO 3 times , but it still won't boot. Are you able to try a power drain, or anything else that might help get it booting again?
Thanks for your time!
The host is depooled and has been banned from the relevant elasticsearch clusters so it's ready to be serviced.
Brian
Banned host like so:
curl -H 'Content-Type: application/json' -XPUT http://localhost:9200/_cluster/settings -d '{"transient":{"cluster.routing.allocation.exclude":{"_host": "","_name": "elastic2033-production-search-codfw"}}}'
curl -H 'Content-Type: application/json' -XPUT http://localhost:9400/_cluster/settings -d '{"transient":{"cluster.routing.allocation.exclude":{"_host": "","_name": "elastic2033-production-search-omega-codfw"}}}'
curl -H 'Content-Type: application/json' -XPUT http://localhost:9600/_cluster/settings -d '{"transient":{"cluster.routing.allocation.exclude":{"_host": "","_name": "elastic2033-production-search-psi-codfw"}}}'Boot was set to UEFI for some reason. I changed it back to Legacy BIOS, system is back online
Mentioned in SAL (#wikimedia-operations) [2022-04-12T22:39:06Z] <ryankemper> T305646 Re-enabling puppet on elastic2033; still need to unban from elasticsearch cluster tomorrow
Cookbook cookbooks.sre.hosts.reimage was started by ryankemper@cumin1001 for host elastic2033.codfw.wmnet with OS stretch
Cookbook cookbooks.sre.hosts.reimage started by ryankemper@cumin1001 for host elastic2033.codfw.wmnet with OS stretch completed:
Mentioned in SAL (#wikimedia-operations) [2022-06-02T19:08:26Z] <ryankemper> T305646 T308647 Unbanned elastic2033 and elastic2054 from clusters; also pooled elastic2033