Page MenuHomePhabricator

Move lvs2012 primary uplink and connect to new row A/B vlans
Closed, ResolvedPublic

Description

In order to support connecting servers to the new, per-rack vlans in rows A and B, we need to move lvs2012's primary uplink so it terminates on lsw1-b2-codfw. Once moved it will be able to reach all row A and B vlans over this link, and we can decom its link to row A.

This change will require co-ordination between DC-Ops, Netops and Traffic teams. This task covers the changes described under "phase one" of the parent task.

Detailed steps are as follows:

1. Pre-configure the new switch port for the connection.

Netops will take care of this. Port xe-0/0/42 needs to be enabled in Netbox and have all the row A and B vlans configured as tagged on it. Untagged vlan will remain private1-b-codfw.

2. Downtime lvs2012, cr1-codfw and cr2-codfw

3. Depool lvs2012

  • Disable puppet
  • Disable pybal
  • Wait until IPVS shows no connections
  • Monitor uptick in connections to lvs2014

4. Connect lvs2012 interface eno12399np0 to lsw1-b2-codfw xe-0/0/42

This is the first port of the first 10G NIC in lvs2012, currently connected via DAC to asw-b2-codfw xe-2/0/45. Should be a simple swap of the switch side from one to the other.

5. Remove optic and fibers from lvs2012 interface eno12409np1 to asw-a7-codfw xe-7/0/45

This is the second port on the first 10G NIC on the host. We can remove this link as we will trunk the row B vlans over eno12399np0 instead.

5. Merge patch to move / add vlan sub-interfaces to lvs2012 eno12399np0

The below patch contains the necessary changes and additions to connect the LB to the required vlans over eno12399np0.

https://gerrit.wikimedia.org/r/c/operations/puppet/+/980931

6. Push new network configuration to lvs2012

Once the above patch is merged we can run puppet on the host, then reboot, for it to take effect. Once the system is back up we should check connectivity on all the vlans by pinging the first IP in each subnet.

7. Default old asw-[a|b]-codfw ports that had been connected to lvs2012

Remove these ports in Netbox and run Homer to apply the changes to the legacy switches (netops).

8. Remove downtime for host and CRs

Event Timeline

Change 980931 had a related patch set uploaded (by Cathal Mooney; author: Cathal Mooney):

[operations/puppet@production] Add new codfw per-rack vlans to lvs2012 and move row A vlans

https://gerrit.wikimedia.org/r/980931

Mentioned in SAL (#wikimedia-operations) [2024-01-16T17:14:16Z] <topranks> Disabling puppet and PyBal on lvs2012 ahead of migration of network link to lsw1-b2-codfw T352909

Change 980931 merged by Cathal Mooney:

[operations/puppet@production] Add new codfw per-rack vlans to lvs2012 and move row A vlans

https://gerrit.wikimedia.org/r/980931

Work completed, all looking good.