Page MenuHomePhabricator
Create Task
Maniphest T200068

labcontrol1003: reimage+rename to cloudcontrol1003
Closed, ResolvedPublic
Actions

Description

Reimage + rename this server to the new naming scheme.

Timeline is:

  • backup db (in /home/aborrero/db.sql @ cloudcontrol1004)
  • disable puppet in labcontrol1003
  • merge puppet patch to rename and get the new debian installer working
  • merge dns patch to add the new FQDNs (partial, the old mgmt names still remains)
  • run the wmf-auto-reimage-host script
  • restore db
  • merge DNS cleanup patch
  • racktables update
  • get the physical relabeling done (T200080)
  • done

Same happened to labvirt1021/cloudvirt1021 (see T199107 and T199132).

Details

SubjectRepoBranchLines +/-
cloudvps: cleanup labcontrol1003 resourcesoperations/dnsmaster+2 -4
cloudvps: rename labnet1003.wikimedia.org to cloudnet1003.wikimedia.orgoperations/puppetproduction+12 -16
cloudvps: add cloudcontrol1003.wikimedia.org FQDNoperations/dnsmaster+6 -4
Revert "cloudvps: eqiad1: use cloudcontrol1004 as primary control server"operations/puppetproduction+8 -8
cloudvps: eqiad1: use cloudcontrol1004 as primary control serveroperations/puppetproduction+8 -8
Customize query in gerrit

Related Objects
Search...

Event Timeline

aborrero triaged this task as Medium priority.Jul 20 2018, 9:09 AM
aborrero created this task.
Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptJul 20 2018, 9:09 AM
gerritbot added a comment.Jul 20 2018, 9:13 AM

Change 447046 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudvps: eqiad1: use cloudcontrol1004 as primary control server

https://gerrit.wikimedia.org/r/447046

gerritbot added a project: Patch-For-Review.Jul 20 2018, 9:13 AM
gerritbot added a comment.Jul 20 2018, 9:16 AM

Change 447046 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudvps: eqiad1: use cloudcontrol1004 as primary control server

https://gerrit.wikimedia.org/r/447046

gerritbot added a comment.Jul 20 2018, 9:28 AM

Change 447048 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] Revert "cloudvps: eqiad1: use cloudcontrol1004 as primary control server"

https://gerrit.wikimedia.org/r/447048

gerritbot added a comment.Jul 20 2018, 9:29 AM

Change 447048 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] Revert "cloudvps: eqiad1: use cloudcontrol1004 as primary control server"

https://gerrit.wikimedia.org/r/447048

gerritbot added a comment.Jul 20 2018, 10:46 AM

Change 447056 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/dns@master] cloudvps: add cloudcontrol1003.wikimedia.org FQDN

https://gerrit.wikimedia.org/r/447056

gerritbot added a comment.Jul 20 2018, 10:46 AM

Change 447056 merged by Arturo Borrero Gonzalez:
[operations/dns@master] cloudvps: add cloudcontrol1003.wikimedia.org FQDN

https://gerrit.wikimedia.org/r/447056

gerritbot added a comment.Jul 20 2018, 10:53 AM

Change 447057 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudvps: rename labnet1003.wikimedia.org to cloudnet1003.wikimedia.org

https://gerrit.wikimedia.org/r/447057

aborrero updated the task description. (Show Details)Jul 20 2018, 10:53 AM
gerritbot added a comment.Jul 20 2018, 10:58 AM

Change 447057 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudvps: rename labnet1003.wikimedia.org to cloudnet1003.wikimedia.org

https://gerrit.wikimedia.org/r/447057

aborrero updated the task description. (Show Details)Jul 20 2018, 11:00 AM
ops-monitoring-bot subscribed.Jul 20 2018, 11:05 AM

Script wmf-auto-reimage was launched by aborrero on neodymium.eqiad.wmnet for hosts:

labcontrol1003.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201807201105_aborrero_1674_labcontrol1003_wikimedia_org.log.

aborrero updated the task description. (Show Details)Jul 20 2018, 11:05 AM
aborrero added a comment.Jul 20 2018, 11:35 AM

Same issue as in some other hosts:

(initramfs) cat /etc/mdadm/mdadm.conf 
HOMEHOST <system>
ARRAY /dev/md/0  metadata=1.2 UUID=83b2d878:ec1733d3:01cd882d:83a54de1 name=cloudcontrol1003:0
ARRAY /dev/md/1  metadata=1.2 UUID=cdff5b81:80a9429c:552e2ad1:0e5918eb name=cloudcontrol1003:1
(initramfs) mdadm --assemble /dev/md/0
mdadm: /dev/md/0 has been started with 4 drives.
(initramfs) exit
ops-monitoring-bot added a comment.Jul 20 2018, 12:11 PM

Completed auto-reimage of hosts:

['cloudcontrol1003.wikimedia.org']

and were ALL successful.

aborrero updated the task description. (Show Details)Jul 20 2018, 12:16 PM
gerritbot added a comment.Jul 20 2018, 12:45 PM

Change 447063 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/dns@master] cloudvps: cleanup labcontrol1003 resources

https://gerrit.wikimedia.org/r/447063

gerritbot added a comment.Jul 20 2018, 12:45 PM

Change 447063 merged by Arturo Borrero Gonzalez:
[operations/dns@master] cloudvps: cleanup labcontrol1003 resources

https://gerrit.wikimedia.org/r/447063

aborrero updated the task description. (Show Details)Jul 20 2018, 12:46 PM
aborrero mentioned this in T200080: Relabel labcontrol1003.wikimedia.org as cloudcontrol1003.wikimedia.org.Jul 20 2018, 12:48 PM
aborrero updated the task description. (Show Details)
• Cmjohnson closed subtask T200080: Relabel labcontrol1003.wikimedia.org as cloudcontrol1003.wikimedia.org as Resolved.Jul 25 2018, 3:05 PM
bd808 added a project: cloud-services-team (Kanban).Jul 26 2018, 4:04 AM
aborrero closed this task as Resolved.Aug 6 2018, 8:51 AM
aborrero updated the task description. (Show Details)

All is done now.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits