Hosts are provisioned and ready to go (T425088).
The one small caveat is that currently reimage must be coordinated as per below
though I don't think we require reimage, nor should we.
Here's what I did yesterday to put cloudvirt1077 in service:
- Make sure the host has no firewall (insetup_noferm) and reimage if needed
- Allocate cloud-private v4/v6 addresses in the host rack vlan, specifically https://netbox.wikimedia.org/ipam/ip-addresses/23745/ and https://netbox.wikimedia.org/ipam/ip-addresses/23746/
- Run the dns cookbook to propagate the names/addresses
- The host can be transitioned to wmcs::openstack::eqiad1::virt_ceph role
- Run puppet with the new role, reboot
- Verify nova-compute is up and running
- On any cloudcontrol host, put the host in service:
- Allocate the host to a cell nova-manage cell_v2 discover_hosts --verbose
- Add the host to ceph and network-ovs aggregates: wmcs-openstack aggregate add host ceph cloudvirt1077 wmcs-openstack aggregate add host network-ovs cloudvirt1077
- Sync the aggregates (TBD if this happens periodically in the background) nova-manage placement sync_aggregates
- Verify the hypervisor shows up: wmcs-openstack hypervisor list | grep 1077
- Verify nova-compute service is enabled, VMs will be scheduled wmcs-openstack compute service list --host cloudvirt1077
- Verify live migration works, pick a test instance, then its server ID with wmcs-server-id <fqdn> and (in this case for pontoon-demo-puppet-01.testlabs.eqiad1.wikimedia.cloud) then run wmcs-openstack server migrate --live --host cloudvirt1077 24e5b10a-80df-4bbc-807c-97d4e935d1f4 --wait and verify the operation is successful (no errors reported, VM stays up and moves to the new host)
- Once the host is in service, hardcode its nova id (/etc/nova/compute_id) in puppet with a patch like https://gerrit.wikimedia.org/r/c/operations/puppet/+/1305668