Page MenuHomePhabricator

Upgrade wmcs OpenStack version to Ocata
Closed, ResolvedPublic

Description

Currently Horizon is running Ocata; all over services are running Newton.

Designate can be upgraded first; the other services are coupled and need to be upgraded together.

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+1 -1
operations/puppetproduction+3 -7
operations/puppetproduction+1 -1
operations/puppetproduction+1 -16
operations/puppetproduction+11 -3
operations/puppetproduction+1 K -64
operations/puppetproduction+66 -1
operations/puppetproduction+5 -4
operations/puppetproduction+1 -1
operations/puppetproduction+5 -3
operations/puppetproduction+2 -3
operations/puppetproduction+23 -0
operations/puppetproduction+99 -0
operations/puppetproduction+2 -2
operations/puppetproduction+2 -0
operations/puppetproduction+24 -18
operations/puppetproduction+426 -1
operations/puppetproduction+8 K -12
Show related patches Customize query in gerrit

Related Objects

StatusSubtypeAssignedTask
ResolvedAndrew
ResolvedAndrew
Resolvedaborrero
Resolvedaborrero
Resolvedaborrero
ResolvedPapaul
Resolved JHedden
Resolvedaborrero
Resolvedaborrero
ResolvedPapaul
Resolvedaborrero
Resolvedaborrero
Resolvedaborrero
Resolvedaborrero
ResolvedAndrew
Resolvedaborrero
Resolvedaborrero
ResolvedAndrew
Resolvedaborrero
Resolvedaborrero
ResolvedAndrew
Resolved Marostegui
Resolvedaborrero
ResolvedAndrew
DuplicateNone
ResolvedAndrew
ResolvedAndrew
Invalid JHedden
Resolvedaborrero
ResolvedAndrew
Resolved Marostegui
ResolvedAndrew

Event Timeline

Andrew created this task.Nov 8 2019, 5:15 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 8 2019, 5:15 PM

Change 550502 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Openstack: Add config files for version 'ocata'

https://gerrit.wikimedia.org/r/550502

Change 550503 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] OpenStack Nova: Update config to work with Ocata

https://gerrit.wikimedia.org/r/550503

Change 552059 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: bootstrap ocata puppet code for servers

https://gerrit.wikimedia.org/r/552059

Change 550502 merged by Andrew Bogott:
[operations/puppet@production] Openstack: Add config files for version 'ocata'

https://gerrit.wikimedia.org/r/550502

Change 552059 merged by Andrew Bogott:
[operations/puppet@production] openstack: bootstrap ocata puppet code for servers

https://gerrit.wikimedia.org/r/552059

Change 550503 merged by Andrew Bogott:
[operations/puppet@production] OpenStack Nova: Update config to work with Ocata

https://gerrit.wikimedia.org/r/550503

Change 552287 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] codfw1dev: move designate to 'Ocata'

https://gerrit.wikimedia.org/r/552287

Change 552287 merged by Andrew Bogott:
[operations/puppet@production] codfw1dev: move designate to 'Ocata'

https://gerrit.wikimedia.org/r/552287

Change 552322 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] codfw1dev: move designate to 'Ocata'

https://gerrit.wikimedia.org/r/552322

Change 552322 merged by Andrew Bogott:
[operations/puppet@production] eqiad1: move designate to 'Ocata'

https://gerrit.wikimedia.org/r/552322

eqiad1 (cloudservices1003/1004) now running ocata Designate.

Change 553877 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Openstack keystone: add more missing manifests for Ocata

https://gerrit.wikimedia.org/r/553877

Change 553878 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Openstack Neutron: add missing files for Openstack Ocata

https://gerrit.wikimedia.org/r/553878

Change 553877 merged by Andrew Bogott:
[operations/puppet@production] Openstack keystone: add more missing manifests for Ocata

https://gerrit.wikimedia.org/r/553877

Change 553878 merged by Andrew Bogott:
[operations/puppet@production] Openstack Neutron: add missing files for Openstack Ocata

https://gerrit.wikimedia.org/r/553878

Change 554199 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Openstack Keystone: update the password_whitelist auth plugin

https://gerrit.wikimedia.org/r/554199

Change 554199 merged by Andrew Bogott:
[operations/puppet@production] Openstack Keystone: update the password_whitelist auth plugin

https://gerrit.wikimedia.org/r/554199

Change 554232 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] keystone wmtotp: update to conform with new ocata parent class

https://gerrit.wikimedia.org/r/554232

Change 554232 merged by Andrew Bogott:
[operations/puppet@production] keystone wmtotp: update to conform with new ocata parent class

https://gerrit.wikimedia.org/r/554232

Change 554233 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Keystone wmtotp: remove unused arg

https://gerrit.wikimedia.org/r/554233

Change 554233 merged by Andrew Bogott:
[operations/puppet@production] Keystone wmtotp: remove unused arg

https://gerrit.wikimedia.org/r/554233

Change 554239 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] wmfkeystonehooks: reconcile some differences with upstream keystone

https://gerrit.wikimedia.org/r/554239

Change 554239 merged by Andrew Bogott:
[operations/puppet@production] wmfkeystonehooks: reconcile some differences with upstream keystone

https://gerrit.wikimedia.org/r/554239

Change 554241 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] keystone ocata: include custom wsgi script

https://gerrit.wikimedia.org/r/554241

Change 554241 merged by Andrew Bogott:
[operations/puppet@production] keystone ocata: include custom wsgi script

https://gerrit.wikimedia.org/r/554241

Change 554423 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Neutron/ocata: update our l3 hacks using the latest ocata base files

https://gerrit.wikimedia.org/r/554423

Change 554423 merged by Andrew Bogott:
[operations/puppet@production] Neutron/ocata: update our l3 hacks using the latest ocata base files

https://gerrit.wikimedia.org/r/554423

Change 554474 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] neutron dhcp: reconcile dhcp_domain and dns_domain

https://gerrit.wikimedia.org/r/554474

Change 554474 merged by Andrew Bogott:
[operations/puppet@production] neutron dhcp: reconcile dhcp_domain and dns_domain

https://gerrit.wikimedia.org/r/554474

Change 554829 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Openstack: move eqiad1 to version 'ocata'

https://gerrit.wikimedia.org/r/554829

Andrew added a comment.EditedDec 5 2019, 12:21 PM

codfw1-dev is running Ocata, and I've scheduled an upgrade window for eqiad1. Approximate steps are...

  • merge https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/556515/, apply on labweb hosts
  • on cloudcontrol1003:
    • nova-manage db online_migrations
  • merge https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/554829/
  • on cloudcontrol1003 and cloudcontrol1004:
    • apply puppet
    • apt-get install nova glance keystone nova-placement-api nova-api nova-conductor
    • apt-get update
    • Clean out keystone files in /etc/apache2/sites-available
    • apply puppet again
    • stop nova-conductor, nova-api, neutron-server
  • on cloudcontrol1003:
    • keystone-manage db_sync
    • nova-manage db sync
    • glance-manage db sync
    • nova-manage api_db sync
    • # nova-manage db online_data_migrations (?)
  • on cloudnet1003 and cloudnet1004:
    • apt-get install python-openstackclient python-openstacksdk neutron-common neutron-dhcp-agent neutron-l3-agent neutron-linuxbridge-agent neutron-metadata-agent python-neutron python-neutron-lib python-neutronclient -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"
    • apt-get upgrade (needed to resolve neutron linuxbridge dependencies)
    • puppet agent -tv
  • on cloudcontrol1003:
    • neutron-db-manage upgrade head
  • on cloudnet1003 and cloudnet1004:
    • puppet agent -tv
    • restart all neutron services
  • on cloudvirts (via cumin):
    • puppet agent -tv
    • apt-get install nova-compute nova-common nova-compute-kvm python-nova python-novaclient neutron-common neutron-linuxbridge-agent python-neutron python-neutron-lib python-neutronclient -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"
    • apt-get upgrade (needed to resolve neutron linuxbridge dependencies)
    • puppet agent -tv
    • service neutron-linuxbridge-agent restart
    • service nova-compute restart
  • merge https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/556516/, apply on labweb hosts

codfw1-dev is running Ocata, and I've scheduled an upgrade window for eqiad1. Approximate steps are...

This sounds right to me. Thanks!

One question, a few new nova services were introduced but I don't see any reference to them in the steps. No steps are required? I'm thinking on the placement and cell stuff. Just double-checking.

Change 554842 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Openstack codfw1dev: everything is ocata now

https://gerrit.wikimedia.org/r/554842

codfw1-dev is running Ocata, and I've scheduled an upgrade window for eqiad1. Approximate steps are...

This sounds right to me. Thanks!

One question, a few new nova services were introduced but I don't see any reference to them in the steps. No steps are required? I'm thinking on the placement and cell stuff. Just double-checking.

you're right, I will add a step about the placement api. Nothing needed for the cells change, that's handled by the existing db sync steps.

Change 554842 merged by Andrew Bogott:
[operations/puppet@production] Openstack codfw1dev: everything is ocata now

https://gerrit.wikimedia.org/r/554842

Change 556515 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Horizon: put in maintenance mode for the newton=>ocata upgrade

https://gerrit.wikimedia.org/r/556515

Change 556516 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Revert "Horizon: put in maintenance mode for the newton=>ocata upgrade"

https://gerrit.wikimedia.org/r/556516

Change 556515 merged by Andrew Bogott:
[operations/puppet@production] Horizon: put in maintenance mode for the newton=>ocata upgrade

https://gerrit.wikimedia.org/r/556515

Change 554829 merged by Andrew Bogott:
[operations/puppet@production] Openstack: move eqiad1 to version 'ocata'

https://gerrit.wikimedia.org/r/554829

Change 556516 merged by Andrew Bogott:
[operations/puppet@production] Revert "Horizon: put in maintenance mode for the newton=>ocata upgrade"

https://gerrit.wikimedia.org/r/556516

Andrew closed this task as Resolved.Dec 12 2019, 12:05 PM

Everything is upgraded and all major functions seem fine.