Page MenuHomePhabricator

labnet1004: reimage+rename to cloudnet1004
Closed, ResolvedPublic

Description

Reimage + rename this server to the new naming scheme.

Timeline would be:

  • failover labnet1004 to cloudnet1003
  • disable puppet in labnet1004
  • merge puppet patch to rename and get the new debian installer working
  • merge dns patch to add the new FQDNs (partial, the old mgmt names still remains)
  • run the wmf-auto-reimage-host script
  • merge DNS cleanup patch
  • racktables update
  • get the physical relabeling done (T199921)
  • done

Same happened to labvirt1021/cloudvirt1021 (see T199107 and T199132).

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 18 2018, 12:12 PM
aborrero@labcontrol1003:~ 2s $ sudo ./neutron-cvps.sh router-list
+--------------------------------------+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+------+
| id                                   | name                | external_gateway_info                                                                                                                                                                  | distributed | ha   |
+--------------------------------------+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+------+
| d93771ba-2711-4f88-804a-8df6fd03978a | cloudinstances2b-gw | {"network_id": "5c9ee953-3a19-4e84-be0f-069b5da75123", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "e4fb2771-a361-4add-ac4e-280cc300c59f", "ip_address": "10.64.22.4"}]} | False       | True |
+--------------------------------------+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+------+
aborrero@labcontrol1003:~ 2s $ sudo ./neutron-cvps.sh l3-agent-list-hosting-router d93771ba-2711-4f88-804a-8df6fd03978a
+--------------------------------------+--------------+----------------+-------+----------+
| id                                   | host         | admin_state_up | alive | ha_state |
+--------------------------------------+--------------+----------------+-------+----------+
| 8af5d8a1-2e29-40e6-baf0-3cd79a7ac77b | cloudnet1003 | True           | :-)   | standby  |
| 96c14889-d181-4d9e-aacb-cd9fae14948c | labnet1004   | True           | :-)   | active   |
+--------------------------------------+--------------+----------------+-------+----------+
aborrero@labnet1004:~ $ sudo puppet agent --disable "arturo - neutron failover to cloudnet1003"
aborrero@labnet1004:~ $ sudo systemctl stop neutron-metadata-agent.service neutron-dhcp-agent.service neutron-l3-agent.service neutron-linuxbridge-agent.service
aborrero@labcontrol1003:~ 2s $ sudo ./neutron-cvps.sh l3-agent-list-hosting-router d93771ba-2711-4f88-804a-8df6fd03978a
+--------------------------------------+--------------+----------------+-------+----------+
| id                                   | host         | admin_state_up | alive | ha_state |
+--------------------------------------+--------------+----------------+-------+----------+
| 8af5d8a1-2e29-40e6-baf0-3cd79a7ac77b | cloudnet1003 | True           | :-)   | active   |
| 96c14889-d181-4d9e-aacb-cd9fae14948c | labnet1004   | True           | xxx   | standby  |
+--------------------------------------+--------------+----------------+-------+----------+

failover is completed

aborrero updated the task description. (Show Details)Jul 18 2018, 12:22 PM
aborrero triaged this task as Normal priority.Jul 18 2018, 12:24 PM
aborrero updated the task description. (Show Details)

Change 446581 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudvps: reimage/rename labnet1004 to cloudnet1004

https://gerrit.wikimedia.org/r/446581

Change 446581 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudvps: reimage/rename labnet1004 to cloudnet1004

https://gerrit.wikimedia.org/r/446581

Change 446582 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/dns@master] cloudvps: rename labnet1004 to cloudnet1004

https://gerrit.wikimedia.org/r/446582

aborrero updated the task description. (Show Details)Jul 18 2018, 12:37 PM

Change 446582 merged by Arturo Borrero Gonzalez:
[operations/dns@master] cloudvps: rename labnet1004 to cloudnet1004

https://gerrit.wikimedia.org/r/446582

aborrero updated the task description. (Show Details)Jul 18 2018, 12:37 PM

Script wmf-auto-reimage was launched by aborrero on neodymium.eqiad.wmnet for hosts:

labnet1004.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201807181244_aborrero_30447_labnet1004_eqiad_wmnet.log.

Completed auto-reimage of hosts:

['cloudnet1004.eqiad.wmnet']

and were ALL successful.

aborrero updated the task description. (Show Details)Jul 18 2018, 3:37 PM

Change 446611 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/dns@master] cloudvps: cleanup labnet1004 resources

https://gerrit.wikimedia.org/r/446611

Change 446611 merged by Arturo Borrero Gonzalez:
[operations/dns@master] cloudvps: cleanup labnet1004 resources

https://gerrit.wikimedia.org/r/446611

aborrero updated the task description. (Show Details)Jul 18 2018, 3:44 PM
aborrero updated the task description. (Show Details)Jul 18 2018, 3:49 PM
aborrero closed this task as Resolved.Aug 6 2018, 12:39 PM
aborrero updated the task description. (Show Details)

Done.