* prepare node for offline actions (but note the idrac update is not offline, its just easier to offline the node and do it all in one offline window)
* offline the node (icinga maint mode, etc)
* screen session on cumin host for cookbooks
* command example: sudo cookbook sre.hardware.upgrade-firmware --component idrac kafka-main1005
** Pick the version just shy of 6.0, 6.0+ has https idrac mgmt issues
** Update will run on the host, flashing to the specified version, and echoing output.
* command example: sudo cookbook sre.hardware.upgrade-firmware --component nic kafka-main1005
** upgrade to 21.85.21.92, as 22.00.07.60 breaks installer
** Update will run on host, etc...
* (optional) command example: sudo cookbook sre.hardware.upgrade-firmware --comonent bios kafka-main1005
** we just always pick latest version of bios available, never had issues.
** Update will run on host, etc...
* Rejoice!