Creating this task to track the work of bringing the new EVPN switches live in codfw row A/B, and making the Spines IP gateway for the existing subnets.
As discussed in the parent task we need to move the links from the row A/B ASWs
| Rack | ASW Side (stays same) | Existing Port | New Port |
|---|---|---|---|
| A8 | asw-a7-codfw et-7/0/52 | cr2-codfw et-1/0/0 | ssw1-a8-codfw et-0/0/29 |
| A8 | asw-b7-codfw et-7/0/52 | cr2-codfw et-1/0/3 | ssw1-a8-codfw et-0/0/30 |
| A1 | asw-a2-codfw et-2/0/52 | cr1-codfw et-1/0/0 | ssw1-a1-codfw et-0/0/29 |
| A1 | asw-b2-codfw et-2/0/51 | cr1-codfw et-1/0/3 | ssw1-a1-codfw et-0/0/30 |
We need to do these 1 at a time in co-ordination between netops and dc-ops. At a high-level the plan will be:
- Move links in rack A8/A1 from CR to SSW
- This brings up new trunk from existing asw vc to the spine, connecting new switches to existing vlans
- We bridge the existing vlans through to the CR which continues to act as IP GW initially
- Migrate IP GW from CRs to SSW
- This ensures traffic arriving from the asw to ssw is optimally forwarded
- Without it the CR VRRP config causes traffic hitting ssw1-a8-codfw for the VRRP MAC to route via a LEAF to ssw1-a1-codfw to get to the active CR
- We keep the CR sub-interfaces connecting to the legacy vlans, so that BGP next-hops are directly reachable
The basic trick when moving the GW IP is to leave VRRP running on the CRs after the first cable move, but change the virtual IP configured. This keeps the VRRP MAC operational, ensuring traffic from hosts with the old IP<->MAC binding still cached in ARP/ND table will be forwarded by the CRs.
To support this we need to change the SSW link to CRs to a layer-2 trunk on the SSW side, and BGP peer to the CRs from an IRB interface over an xlink vlan. This will allow us to trunk the existing Vlans to the CRs on the same link, so that the CRs can retain a direct connect to these networks. This ensures the CRs can still route to VIPs announced by end-hosts in BGP, which currently peer with the CR loopbacks. Without a direct leg in the Vlan the peering would work, but the routing would break as the EVPN switches do not know those routes.