Page MenuHomePhabricator

Upgrade cloudvirts to Debian Buster
Closed, ResolvedPublic

Description

It ought to be possible to upgrade the ceph-enabled cloudvirts immediately, and then the others as they're drained and converted to ceph.

Related Objects

StatusSubtypeAssignedTask
ResolvedMajavah
ResolvedAndrew
ResolvedAndrew
Resolveddcaro
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedBstorm
OpenNone
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
Opendcaro
Resolvedaborrero
Opendcaro
Resolveddcaro
Opendcaro
Opendcaro
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedCmjohnson
Resolvedayounsi
Resolvedaborrero
ResolvedCmjohnson
ResolvedJclark-ctr
OpenCmjohnson

Event Timeline

Change 617791 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] openstack::nova::compute::service::rocky::buster: use legacy ebtables

https://gerrit.wikimedia.org/r/617791

Change 617791 merged by Andrew Bogott:
[operations/puppet@production] openstack::nova::compute::service::rocky::buster: use legacy ebtables

https://gerrit.wikimedia.org/r/617791

andrew@cumin1001:~$ sudo cumin --force --timeout 500 "cloudvirt1*" "lsb_release -sc"
37 hosts will be targeted:
cloudvirt[1001-1009,1012-1039].eqiad.wmnet
FORCE mode enabled, continuing without confirmation

NODE GROUP

(7) cloudvirt[1033-1039].eqiad.wmnet

  • OUTPUT of 'lsb_release -sc' -----

buster

NODE GROUP

(30) cloudvirt[1001-1009,1012-1032].eqiad.wmnet

  • OUTPUT of 'lsb_release -sc' -----

stretch

Change 618124 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudvirt1031, 1032 -> buster

https://gerrit.wikimedia.org/r/618124

Change 618124 merged by Andrew Bogott:
[operations/puppet@production] cloudvirt1031, 1032 -> buster

https://gerrit.wikimedia.org/r/618124

Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for hosts:

['cloudvirt1032.eqiad.wmnet', 'cloudvirt1031.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202008031758_andrew_23683.log.

Completed auto-reimage of hosts:

['cloudvirt1032.eqiad.wmnet', 'cloudvirt1031.eqiad.wmnet']

Of which those FAILED:

['cloudvirt1032.eqiad.wmnet', 'cloudvirt1031.eqiad.wmnet']

Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for hosts:

['cloudvirt1032.eqiad.wmnet', 'cloudvirt1031.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202008101357_andrew_25207.log.

Completed auto-reimage of hosts:

['cloudvirt1031.eqiad.wmnet', 'cloudvirt1032.eqiad.wmnet']

Of which those FAILED:

['cloudvirt1031.eqiad.wmnet', 'cloudvirt1032.eqiad.wmnet']

Change 619315 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudvirt103[1,2]: update nic names for Buster

https://gerrit.wikimedia.org/r/619315

Change 619315 merged by Andrew Bogott:
[operations/puppet@production] cloudvirt103[1,2]: update nic names for Buster

https://gerrit.wikimedia.org/r/619315

Andrew triaged this task as Medium priority.Aug 11 2020, 4:41 PM
NOTE: I tested ebtables-nft 1.8.3 from buster-bpo and it doesn't work. We would need 1.8.5 per iptables upstream changelog (among support).

Change 630656 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Move cloudvirt1012,13 and 14 to ceph and Buster

https://gerrit.wikimedia.org/r/630656

Change 630656 merged by Andrew Bogott:
[operations/puppet@production] Move cloudvirt1012,13 and 14 to ceph and Buster

https://gerrit.wikimedia.org/r/630656

Change 630712 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Update nic names for cloudvirt1012, 1013, 1014

https://gerrit.wikimedia.org/r/630712

Change 630712 merged by Andrew Bogott:
[operations/puppet@production] Update nic names for cloudvirt1012, 1013, 1014

https://gerrit.wikimedia.org/r/630712

Change 630941 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Cloudvirt1016 -> Buster/Ceph

https://gerrit.wikimedia.org/r/630941

Change 630941 merged by Andrew Bogott:
[operations/puppet@production] Cloudvirt1016 -> Buster/Ceph

https://gerrit.wikimedia.org/r/630941

Change 631325 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudvirt1017 to Buster and Ceph

https://gerrit.wikimedia.org/r/631325

Change 631325 merged by Andrew Bogott:
[operations/puppet@production] cloudvirt1017 to Buster and Ceph

https://gerrit.wikimedia.org/r/631325

Change 631450 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudvirt-wdqs* to Buster

https://gerrit.wikimedia.org/r/631450

Change 631450 merged by Andrew Bogott:
[operations/puppet@production] cloudvirt-wdqs* to Buster

https://gerrit.wikimedia.org/r/631450

Change 631541 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudvirt1018 to Buster/Ceph

https://gerrit.wikimedia.org/r/631541

Change 631541 merged by Andrew Bogott:
[operations/puppet@production] cloudvirt1018 to Buster/Ceph

https://gerrit.wikimedia.org/r/631541

Change 632296 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Cloudvirt1021 and 1022 to Buster and Ceph

https://gerrit.wikimedia.org/r/632296

Change 632296 merged by Andrew Bogott:
[operations/puppet@production] Cloudvirt1021 and 1022 to Buster and Ceph

https://gerrit.wikimedia.org/r/632296

Change 632545 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudvirt1023: to Ceph and Buster

https://gerrit.wikimedia.org/r/632545

Change 632545 merged by Andrew Bogott:
[operations/puppet@production] cloudvirt1023: to Ceph and Buster

https://gerrit.wikimedia.org/r/632545

Change 632972 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudvirt1023: update nic names

https://gerrit.wikimedia.org/r/632972

Change 632972 merged by Andrew Bogott:
[operations/puppet@production] cloudvirt1023: update nic names

https://gerrit.wikimedia.org/r/632972

Change 635680 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Cloudvirt1025 and 1029 => Buster

https://gerrit.wikimedia.org/r/635680

Change 635680 merged by Andrew Bogott:
[operations/puppet@production] Cloudvirt1025 and 1029 => Buster

https://gerrit.wikimedia.org/r/635680

Change 635858 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Remaining eqiad1 cloudvirts to Buster

https://gerrit.wikimedia.org/r/635858

Change 635858 merged by Andrew Bogott:
[operations/puppet@production] Remaining eqiad1 cloudvirts to Buster

https://gerrit.wikimedia.org/r/635858

Change 636507 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Cloudvirt1027/1028 to ceph and backy2

https://gerrit.wikimedia.org/r/636507

Change 636507 merged by Andrew Bogott:
[operations/puppet@production] Cloudvirt1027/1028 to ceph and backy2

https://gerrit.wikimedia.org/r/636507

Change 644943 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Cloudvirt1025-1030: Update nic names for 10Gb/Buster

https://gerrit.wikimedia.org/r/644943

Change 644943 merged by Andrew Bogott:
[operations/puppet@production] Cloudvirt1025-1030: Update nic names for 10Gb/Buster

https://gerrit.wikimedia.org/r/644943

All cloudvirts are now running Buster save cloudvirt1019, which is T263677.