Similar to the work done under T420180 this task will focus on draining traffic through ssw1-d8-eqiad (and thus via cr2-eqiad), so we can proceed and reset BGP sessions from Nokia leaf switches in rows C/D which have an invalid vxlan tunnel id to it, triggering the DHCP bug described in the parent task.

Process

• Set cr1 to be VRRP master for all vlans P89874
  • This will ensure row a/b hosts send traffic to cr1, which will then route to ssw1-d1
  • It will ensure that only ssw1-d1 learns the VRRP GW MAC for the row c/d vlans, so leaf switches will not see route to it from ssw1-d8
• Disable VRRP for row-wide vlan sub-interfaces of cr2-eqiad et-1/0/5 - P89872
  • This is needed as we don't want to create a VRRP "split brain" scenario (P89795)
• Increase the OSPF cost on the far-side of all transport links terminating on cr2
  • This will ensure traffic from other sites to row c/d vlans should instead arrive on cr1, and take path out via ssw1-d1
• Set 'graceful-shutdown' for all server BGP groups on cr2-eqiad, to lower local pref for routes learnt P89873
  • This will make cr2 prefer the routes hosts are announcing to cr1 via the IBGP route
• Also set 'graceful-shutdown' for internet-facing BGP groups on cr2-eqiad and pre-pend out P89873
  • This will shift incoming traffic for the public1-c-eqiad and public1-d-eqiad ranges to come in on cr1, and use ssw1-d1
• Disable the EVPN IBGP peering between ssw1-d8 and ssw1-d1:
  • ssw1-d8: set / network-instance default protocols bgp neighbor 10.64.128.17 admin-state disable
  • This ensures that ssw1-d1 does not reflect routes from ssw1-d8 to leafs
  • Which means clearing ssw1-d8 BGP session to leaf will remove all routes using it as next-hop
• Adjust the ssw1-d8 BGP config to not accept or announce any routes to cr2 or other row e/f spines
  • By changing the import/export policies to 'NONE' - P89816
• Adjust the cr2 BGP policy for row e/f and cloudsw to not export directly connected routes
  • cr2-eqiad: delete policy-options policy-statement Switch_out term direct
  • This ensures no L3 switches will use cr2 to get to row c/d vlans, instead they will use cr1 uplink

Once complete we should be able to observe there is no traffic on cr2-eqiad et-1/0/5. Provided that is the case we can proceed on the leaf switches in rows C and D and reset the BGP peering to ssw1-d8-eqiad without interrupting traffic flows.