Once T405628 is complete we can look at this task.
Effectively there are two objectives:
- Reimage the host, moving it from the legacy row-wide vlan to the per-rack one available on the new switch
- Adjust the BGP config to peer with the switch directly, instead of with the CR routers
The real benefit of this is that the switches have a specific route for all the VIPs it announces, and thus traffic coming from our other hosts to services behind the load-balancer will take an optimal path through the network, avoiding the bottleneck of our core routers if it originates within row C or D.
Basic steps should be:
- Create puppet patch so the LVS peers with the .1 address on the new vlan (the switch itself)
- The puppet patch should also add config for private1-c-eqiad to be a new vlan sub-interface of the primary
- Downtime the host
- Set the 'bgp' flag to false in Netbox for the host
- Run Homer against cr1-eqiad, cr2-eqiad to remove the BGP config for the host on the CRs
- Observe live stats to verify that traffic re-directs to the backup lvs1020 without an issue, and connections go to zero
- Kick off a reimage of the host with the --move-vlan option set
- Once the cookbook has edited the Netbox config, manually check all required vlans are still on the primary port
- If not add them back in, --move-vlan probably is not designed to deal with those
- Also add private1-c-eqiad
- Merge the puppet patch while the OS is installing
- When it comes back verify the config looks ok and setup is good
- Run Homer against lsw1-c7-eqiad to add the BGP config and trunk private1-c-eqiad on the port
- Validate BGP is back up, routes announced
- Check that connections start rising and it takes over from the backup successfully