We previously consolidated on 4.4.0-81-generic #104~14.04.1-Ubuntu SMP Wed Jun 14 12:45:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux because of issues with varying kernel versions. That seemed like a safe choice, and I thought we ran through every labvirt at the time. But now I'm not so sure.
Reproduce:
ssh sarin.codfw.wmnet
screen /home/rush/wmf-auto-reimage-stop-after-puppet -d labtestvirt2003.codfw.wmnet labtestvirt2003.mgmt.codfw.wmnet
(slightly hacked version that stops before the puppet run. Wait for it to exit.)
ssh puppetmaster1001.eqiad.wmnet
install-console labtestvirt2003.codfw.wmnet
uname -a (see 3.13 kernel)
aptitude install linux-image-4.4.0-81-generic
passwd
(set the root password now because w/o a puppet run to set it and no network this will come back blind)
/sbin/reboot
It will be dead except via console on 4.4
login via console
ip link show
(has no network interfaces)
aptitude remove --purge linux-image-4.4.0-81-generic
(removes 4.4 after which it will reboot into 3.13 and be back on the network)
https://wikitech.wikimedia.org/wiki/Platform-specific_documentation/HP_DL3N0#Common_Actions