Page MenuHomePhabricator

Upgrade eqiad1 cloudservices nodes to OpenStack 'Wallaby'
Closed, ResolvedPublic

Description

The release notes (https://docs.openstack.org/releasenotes/designate/) for the change between V and W are essentially empty, so this should be an easy one.

  • prepare a patch that upgrades the openstack version /only/ on cloudservices (via profile::openstack::<deployent>::version settings in puppet/hieradata/hosts/<hostnames>)

https://gerrit.wikimedia.org/r/c/operations/puppet/+/774523

  • disable puppet on cloudservices nodes from cumin1001.eqiad.wmnet

    $ sudo cumin 'cloudservices100*' "disable-puppet 'Upgrading to openstack Wallaby - T304880 - ${USER}'"
  • merge above patch

One node at a time:

  • cloudservices1003.wikimedia.org
  • cloudservices1004.wikimedia.org
  • puppet agent --enable && puppet agent -tv
  • apt-get update
  • DEBIAN_FRONTEND=noninteractive apt-get install python3-designate designate-sink designate-central designate-worker designate-mdns designate-api -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"
  • DEBIAN_FRONTEND=noninteractive apt-get upgrade -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"
  • puppet agent -tv
  • designate-manage database sync
  • puppet agent -tv
  • reboot host
  • systemctl list-units --failed (should show nothing failed)

Test by digging @ both the pdns-recursor ip and the pdns auth ip, a couple of randomly-chosen VM fqdns.
for cloudservices1003.wikimedia.org:
dig @ns0.openstack.eqiad1.wikimediacloud.org paws-k8s-control-1.paws.eqiad1.wikimedia.cloud

for cloudservices1004.wikimedia.org:
dig @ns1.openstack.eqiad1.wikimediacloud.org paws-k8s-control-1.paws.eqiad1.wikimedia.cloud

Details

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 774523 had a related patch set uploaded (by Vivian Rook; author: Vivian Rook):

[operations/puppet@production] Update eqiad1 cloudservices openstack

https://gerrit.wikimedia.org/r/774523

Change 774523 merged by Vivian Rook:

[operations/puppet@production] Update eqiad1 cloudservices openstack

https://gerrit.wikimedia.org/r/774523

cloudservices1003.wikimedia.org is done

cloudservices1004.wikimedia.org is done