Page MenuHomePhabricator

Upgrade firmware on wdqs1009
Closed, ResolvedPublic

Description

wdqs1009 crashed around Feb 13 15:41:01. Syslog and System Event Log don't show anything obviously suspicious. @MoritzMuehlenhoff suggested updating the firmware, which might resolve the issue and would probably be the first step if we need to report this to the vendor.

Event Timeline

@Gehel updating firmware requires rebooting a few times. Let me know if this will be okay, if it needs to be a scheduled downtime, Let's plan for tomorrow Wednesday at 1400UTC or Thursday same time

@Gehel updating firmware requires rebooting a few times. Let me know if this will be okay, if it needs to be a scheduled downtime, Let's plan for tomorrow Wednesday at 1400UTC or Thursday same time

We have a data reload in progress at the moment, I'd prefer not to interrupt it (T267927). I'll ping you (or @RKemper will) once it's done. This is a test server, so once the data reload is done, we can shutdown that server whenever, with no user impact.

@Cmjohnson The data reload is complete on wdqs1009, so the host can now have its firmware upgraded and be rebooted at its convenience. Note this is an internal wdqs test host, so there is no public-facing service for us to worry about.

Feel free to proceed when convenient for you - just shoot me a ping or drop a line in #wikimedia-discovery, but no need to schedule a formal window.

Note that wdqs1011 had a similar issue today (might not be related at all)

Mentioned in SAL (#wikimedia-operations) [2021-03-11T15:53:13Z] <cmjohnson1> updating firmware wdqs1009 T274751

updated BIOS, IDRAC and NIC firmware. Resolving the task, if an issue persists please open a new task with the error.