Due to the issues noted on T376737, we're going to have to reimage a large number of #ops-magru hosts. This task will have the per host checklist for each item and what needs to be checked for each to return to service.
== Migration Step Checklist ==
Rob has compiled the following order of operation after day 1 server swaps. Completing these in any other order may not result in two hosts being ready for reimage:
* Support Swaps each host pair around, Rob remotely confirms hardware is correct and remotely accessible.
* Take screen shots / note the network port and cable ID info for both host A and B.
* Run decommission cookbook for both hosts A and B.
* Pull up both A and B in netbox, swap around A and B hostname + rack location, set both from Decommissioning to Planned state.
* Pull up host A's mgmt ip and enter udpated fqdn under dns name for the mgmt ip entry.
* Run Provision Host Network details for host A, use host B's netowkr port and cable info recorded earlier.
* Pull up host B's mgmt ip and enter udpated fqdn under dns name for the mgmt ip entry.
* Run Provision Host Network details for host B, use host B's netowkr port and cable info recorded earlier.
* Run sre.dns.netbox to push updated dns for hosts A and B.
* Run sre.network.configure-switch-interfaces for host A and host B
== Per host setup checklist ==
==== dns7001
[x] Ensure host is fully migrated per T376737
[x] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[x] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[x] **Immediately** run the `sre.dns.netbox` cookbook
[x] **Immediately** run the `sre.hosts.provision` cookbook
[] NIC doesn't detect, remote hands task updated with info, don't check this line off until NIC is reseated and detects
[] Run the `sre.hosts.reimage` cookbook
==== dns7002
[] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
==== ganeti7001
[] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[] Update the `operations/puppet` repo - this should include updates to preseed.yaml, and site.pp with roles defined by service group: https://wikitech.wikimedia.org/wiki/SRE/Dc-operations
[] Run the `sre.hosts.reimage` cookbook
[] create and link in service implementation task for Mortiz for ganeti700[1-4]
==== ganeti7002
[] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[] Update the `operations/puppet` repo - this should include updates to preseed.yaml, and site.pp with roles defined by service group: https://wikitech.wikimedia.org/wiki/SRE/Dc-operations
[] Run the `sre.hosts.reimage` cookbook
[] create and link in service implementation task for Mortiz for ganeti700[1-4]
==== ganeti7003
[x] Ensure host is fully migrated per T376737
[x] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[x] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[x] **Immediately** run the `sre.dns.netbox` cookbook
[x] **Immediately** run the `sre.hosts.provision` cookbook
[x] Run the `sre.hosts.reimage` cookbook
==== ganeti7004
[x] Ensure host is fully migrated per T376737
[x] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[x] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[x] **Immediately** run the `sre.dns.netbox` cookbook
[x] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
==== lvs7001
[] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
==== lvs7003
[x] Ensure host is fully migrated per T376737
[x] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[x] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[x] **Immediately** run the `sre.dns.netbox` cookbook
[x] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
lvs7003 network info:
nic port 0: cable 70100 : asw1-b3-magru:et-0/0/12
nic port 1: cable 70113 : asw2-b4-magru:et-0/0/13
==== cp7001
[x] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[x] Run the `sre.hosts.reimage` cookbook
==== cp7002
[] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
==== cp7003
[] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
==== cp7004
[] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
==== cp7006
[x] Ensure host is fully migrated per T376737
[x] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[x] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[x] **Immediately** run the `sre.dns.netbox` cookbook
[x] **Immediately** run the `sre.hosts.provision` cookbook
[x] Run the `sre.hosts.reimage` cookbook
==== cp7008
[x] Ensure host is fully migrated per T376737
[x] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[x] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[x] **Immediately** run the `sre.dns.netbox` cookbook
[x] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
==== cp7010
[] Ensure host is fully migrated per T376737
[] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[] **Immediately** run the `sre.dns.netbox` cookbook
[] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook
==== cp7015
[x] Ensure host is fully migrated per T376737
[x] Update netbox with host's new location and service tag, copy down old hostname cable IDs, delete off old hostname, and install via script onto this host.
[x] Run the [[ https://netbox.wikimedia.org/extras/scripts/provision_server.ProvisionServerNetwork/ | Provision a server's network attributes ]] Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
[x] **Immediately** run the `sre.dns.netbox` cookbook
[x] **Immediately** run the `sre.hosts.provision` cookbook
[] Run the `sre.hosts.reimage` cookbook