Page MenuHomePhabricator

Audit/update NIC firmware on Search Platform-owned Buster hosts
Closed, ResolvedPublic3 Estimated Story Points

Description

As Buster is being deprecated, we will need to reimage all our hosts to Bullseye or later. However, before we can reimage the hosts, we need to update their NIC firmware as earlier versions do not support PXE boot.

See T312298 for an example of similar work and this repo for an example of how to do this with minimum disruption.

Event Timeline

MPhamWMF moved this task from needs triage to Ops / SRE on the Discovery-Search board.

Results of audit pasted here . We'll need to update the firmware of those hosts and reboot.

Mentioned in SAL (#wikimedia-operations) [2023-05-17T19:41:21Z] <inflatador> bking@wdqs2012 depooling to attempt firmware update T331297

Host rebooted by bking@cumin1001 with reason: None

Icinga downtime and Alertmanager silence (ID=401c16a2-5570-4b24-b856-a1d2685f312c) set by bking@cumin1001 for 4:00:00 on 1 host(s) and their services with reason: firmware update

wdqs1014.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-06-05T18:32:21Z] <inflatador> bking@cumin1001 depooling wdqs2010 for fw update T331297

Host rebooted by bking@cumin1001 with reason: None

Mentioned in SAL (#wikimedia-operations) [2023-06-05T18:48:03Z] <inflatador> bking@cumin1001 repooling wdqs2010 T331297

Mentioned in SAL (#wikimedia-operations) [2023-06-05T18:48:36Z] <inflatador> bking@cumin1001 depooling wdqs2011for fw update T331297

Host rebooted by bking@cumin1001 with reason: None

bking claimed this task.

All hosts have been updated to recommended firmware version 21.85.21.92 . Moving to needs-reporting...