Page MenuHomePhabricator
Create Task
Maniphest T209426

Rename labvirt1016 to cloudvirt1016, move to eqiad1
Closed, ResolvedPublic
Actions

Description

This host is now empty and ready to move to the new region.

Reimage + rename this server to the new naming scheme.

Timeline would be:

  • disable puppet in labvirt1016
  • merge puppet patch to rename, get the new debian installer working and disable notifications (rename hieradata/hosts/labvirt1016.yaml to cloudvirt1016.yaml and add "profile::base::notifications: disabled" temporarily)
  • merge dns patch to add the new FQDNs (partial, the old mgmt names still remains)
  • run the wmf-auto-reimage-host script (used old-school method)
  • merge DNS cleanup patch
  • merge puppet patch to re-enable notifications (remove "profile::base::notifications")
  • netbox update
  • update docs https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Deployments
  • physical relabeling and switch port description (T209427)
  • done

Details

SubjectRepoBranchLines +/-
cloudvps: Add cloudvirt1016 to the scheduler pooloperations/puppetproduction+2 -1
cloudvps: cleanup labvirt1016operations/dnsmaster+1 -2
cloudvps: hieradata for cloudvirt1016operations/puppetproduction+0 -0
cloudvps: rename+reimage labvirt1016 as cloudvirt1016operations/dnsmaster+5 -4
cloudvps: reimage+rename labvirt1016 as cloudvirt1016operations/puppetproduction+11 -11
Customize query in gerrit

Related Objects
Search...

Event Timeline

Andrew created this task.Nov 13 2018, 10:40 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 13 2018, 10:40 PM
Andrew mentioned this in T209427: Relabel labvirt1016.eqiad.wmnet as cloudvirt1016.eqiad.wmnet.Nov 13 2018, 10:41 PM
Andrew added a project: cloud-services-team (Kanban).
• GTirloni claimed this task.Nov 14 2018, 9:58 AM
• GTirloni triaged this task as Medium priority.
gerritbot subscribed.Nov 14 2018, 3:33 PM

Change 473540 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/puppet@production] cloudvps: reimage+rename labvirt1016 as cloudvirt1016

https://gerrit.wikimedia.org/r/473540

gerritbot added a project: Patch-For-Review.Nov 14 2018, 3:33 PM

Change 473541 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/dns@master] cloudvps: rename+reimage labvirt1016 as cloudvirt1016

https://gerrit.wikimedia.org/r/473541

gerritbot added a comment.Nov 14 2018, 3:36 PM

Change 473540 merged by GTirloni:
[operations/puppet@production] cloudvps: reimage+rename labvirt1016 as cloudvirt1016

https://gerrit.wikimedia.org/r/473540

gerritbot added a comment.Nov 14 2018, 3:37 PM

Change 473541 merged by GTirloni:
[operations/dns@master] cloudvps: rename+reimage labvirt1016 as cloudvirt1016

https://gerrit.wikimedia.org/r/473541

gerritbot added a comment.Nov 14 2018, 4:27 PM

Change 473557 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/puppet@production] cloudvps: hieradata for cloudvirt1016

https://gerrit.wikimedia.org/r/473557

gerritbot added a comment.Nov 14 2018, 4:28 PM

Change 473557 merged by GTirloni:
[operations/puppet@production] cloudvps: hieradata for cloudvirt1016

https://gerrit.wikimedia.org/r/473557

gerritbot added a comment.Nov 14 2018, 4:34 PM

Change 473560 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/dns@master] cloudvps: cleanup labvirt1016

https://gerrit.wikimedia.org/r/473560

gerritbot added a comment.Nov 14 2018, 4:34 PM

Change 473560 merged by GTirloni:
[operations/dns@master] cloudvps: cleanup labvirt1016

https://gerrit.wikimedia.org/r/473560

• GTirloni updated the task description. (Show Details)Nov 14 2018, 4:48 PM
• GTirloni updated the task description. (Show Details)Nov 14 2018, 4:59 PM
• GTirloni updated the task description. (Show Details)Nov 14 2018, 5:01 PM
• GTirloni updated the task description. (Show Details)
gerritbot added a comment.Nov 14 2018, 5:13 PM

Change 473567 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/puppet@production] cloudvps: Add cloudvirt1016 to the scheduler pool

https://gerrit.wikimedia.org/r/473567

• GTirloni added a comment.EditedNov 14 2018, 5:13 PM

Successfully scheduled canary1016-01.testlabs.eqiad.wmflabs on cloudvirt1016.

cloudcontrol1003# OS_PROJECT_ID=testlabs openstack server create --flavor m1.small --image c6273cce-9b8b-4364-9f1f-7bf58436994f --nic net-id=lan-flat-cloudinstances2b --availability-zone host:cloudvirt1016 canary1016-01
gerritbot added a comment.Nov 14 2018, 5:16 PM

Change 473567 merged by GTirloni:
[operations/puppet@production] cloudvps: Add cloudvirt1016 to the scheduler pool

https://gerrit.wikimedia.org/r/473567

Stashbot subscribed.Nov 14 2018, 5:19 PM

Mentioned in SAL (#wikimedia-cloud) [2018-11-14T17:19:34Z] <gtirloni> added cloudvirt1016 to scheduler pool (T209426)

• Cmjohnson closed subtask T209427: Relabel labvirt1016.eqiad.wmnet as cloudvirt1016.eqiad.wmnet as Resolved.Nov 26 2018, 5:12 PM
Andrew closed this task as Resolved.Nov 26 2018, 7:25 PM
Andrew updated the task description. (Show Details)
Dzahn subscribed.Nov 30 2018, 12:33 AM

19:28 <+icinga-wm> PROBLEM - nova-compute proc minimum on labvirt1016 is CRITICAL: NRPE: Command check_ensure_nova_compute_running not defined
19:28 <+icinga-wm> PROBLEM - ensure kvm processes are running on labvirt1016 is CRITICAL: NRPE: Command check_ensure_running_kvm_instances not defined

? that just started and was surprising because this ticket seems to describe the opposite direction of renaming and was closed more than 3 days ago

Andrew added a comment.Nov 30 2018, 12:35 AM

It looks like the deactivate and clean stages were missed during this move. Doing them now.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL