eqord is part of the eqiad confederation, despite being closer to codfw in term of latency. This is a snowflake we can't get rid of, but could improve.
With T246721 we made eqord advertise only eqiad prefixes, and share the prefixes it learns only to eqiad (same confed).
Later on we made eqord advertise codfw prefixes as well (because closer to codfw).
To make eqord share the external prefixes it learned to codfw, without sharing the external prefixes learned from eqiad we have 2 options:
- Use BGP policies
- Separate eqord to its own confed
The 2nd option seems to be the cleaner on the overall design (there is no real reason it's part of eqiad).
Rough steps would be:
- Disable external BGP sessions
- Add the new AS# to confederation 14907 members  (eg. 65020 to not impact future sites numbering).
- Optionally, decide if we want to follow the physical links or keep the kinda full mesh we have -> keep the status quo
- Convert eqiad-eqord iBGP sessions to eBGP (with an policy allowing to export its DFZ, but not importing any DFZ)
- Renumber codfw/ulsfo neighbor AS
- Enable external BGP sessions
- Check that sessions are established and proper prefixes exchanged