Page MenuHomePhabricator

Rename labvirt1016 to cloudvirt1016, move to eqiad1
Closed, ResolvedPublic

Description

This host is now empty and ready to move to the new region.

Reimage + rename this server to the new naming scheme.

Timeline would be:

  • disable puppet in labvirt1016
  • merge puppet patch to rename, get the new debian installer working and disable notifications (rename hieradata/hosts/labvirt1016.yaml to cloudvirt1016.yaml and add "profile::base::notifications: disabled" temporarily)
  • merge dns patch to add the new FQDNs (partial, the old mgmt names still remains)
  • run the wmf-auto-reimage-host script (used old-school method)
  • merge DNS cleanup patch
  • merge puppet patch to re-enable notifications (remove "profile::base::notifications")
  • netbox update
  • update docs https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Deployments
  • physical relabeling and switch port description (T209427)
  • done

Event Timeline

Andrew created this task.Nov 13 2018, 10:40 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 13 2018, 10:40 PM
GTirloni claimed this task.Nov 14 2018, 9:58 AM
GTirloni triaged this task as Normal priority.

Change 473540 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/puppet@production] cloudvps: reimage+rename labvirt1016 as cloudvirt1016

https://gerrit.wikimedia.org/r/473540

Change 473541 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/dns@master] cloudvps: rename+reimage labvirt1016 as cloudvirt1016

https://gerrit.wikimedia.org/r/473541

Change 473540 merged by GTirloni:
[operations/puppet@production] cloudvps: reimage+rename labvirt1016 as cloudvirt1016

https://gerrit.wikimedia.org/r/473540

Change 473541 merged by GTirloni:
[operations/dns@master] cloudvps: rename+reimage labvirt1016 as cloudvirt1016

https://gerrit.wikimedia.org/r/473541

Change 473557 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/puppet@production] cloudvps: hieradata for cloudvirt1016

https://gerrit.wikimedia.org/r/473557

Change 473557 merged by GTirloni:
[operations/puppet@production] cloudvps: hieradata for cloudvirt1016

https://gerrit.wikimedia.org/r/473557

Change 473560 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/dns@master] cloudvps: cleanup labvirt1016

https://gerrit.wikimedia.org/r/473560

Change 473560 merged by GTirloni:
[operations/dns@master] cloudvps: cleanup labvirt1016

https://gerrit.wikimedia.org/r/473560

GTirloni updated the task description. (Show Details)Nov 14 2018, 4:48 PM
GTirloni updated the task description. (Show Details)Nov 14 2018, 4:59 PM
GTirloni updated the task description. (Show Details)Nov 14 2018, 5:01 PM
GTirloni updated the task description. (Show Details)

Change 473567 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/puppet@production] cloudvps: Add cloudvirt1016 to the scheduler pool

https://gerrit.wikimedia.org/r/473567

GTirloni added a comment.EditedNov 14 2018, 5:13 PM

Successfully scheduled canary1016-01.testlabs.eqiad.wmflabs on cloudvirt1016.

cloudcontrol1003# OS_PROJECT_ID=testlabs openstack server create --flavor m1.small --image c6273cce-9b8b-4364-9f1f-7bf58436994f --nic net-id=lan-flat-cloudinstances2b --availability-zone host:cloudvirt1016 canary1016-01

Change 473567 merged by GTirloni:
[operations/puppet@production] cloudvps: Add cloudvirt1016 to the scheduler pool

https://gerrit.wikimedia.org/r/473567

Mentioned in SAL (#wikimedia-cloud) [2018-11-14T17:19:34Z] <gtirloni> added cloudvirt1016 to scheduler pool (T209426)

Andrew closed this task as Resolved.Nov 26 2018, 7:25 PM
Andrew updated the task description. (Show Details)
Dzahn added a subscriber: Dzahn.Nov 30 2018, 12:33 AM

19:28 <+icinga-wm> PROBLEM - nova-compute proc minimum on labvirt1016 is CRITICAL: NRPE: Command check_ensure_nova_compute_running not defined
19:28 <+icinga-wm> PROBLEM - ensure kvm processes are running on labvirt1016 is CRITICAL: NRPE: Command check_ensure_running_kvm_instances not defined

? that just started and was surprising because this ticket seems to describe the opposite direction of renaming and was closed more than 3 days ago

It looks like the deactivate and clean stages were missed during this move. Doing them now.