Change Details

Looking at some stats I see we barely use our Lumen transit in Codfw. The reason is that most prefixes we see with the same AS-path length from Arelion and Lumen. But Arelion have MED=0 on the routes they send, and Lumen have various values higher than that, which is getting used as a tie-break, for example: ``` cmooney@re0.cr2-codfw> show route protocol bgp 176.61.33.2 terse inet.0: 937054 destinations, 3246024 routes (936878 active, 61 holddown, 419 hidden) Restart Complete + = Active Route, - = Last Active, * = Both A V Destination P Prf Metric 1 Metric 2 Next hop AS path * V 176.61.0.0/18 B 170 100 0 1299 6830 ? valid >208.80.153.218 208.80.153.213 V B 170 100 22640 3356 6830 ? valid >64.156.73.169 ``` We also had the recent complications from accepting MED inbound from Amazon in eqord and eqiad and how we use it internally. We can wait here for the larger evaluation, but I thought it might be worth discussing here either way as it's part of the equation. From my part I don't believe it is serving us well to tie-break on this attribute. My logic being: * We should ultimately control the policy about where we send outbound traffic * Different carriers have different policies, like this between Lumen and Arelion ** Whatever Lumen may be trying to signal, it's definitely not "use Arelion instead" * We should be doing hot-potato routing from our sites towards the internet ** So the only valid MED comparison would be between two routes from the same carrier at the same site I think it would be reasonable, and give us more control plus make things easier to reason about, to just set MED=0 on all inbound routes so it is not used as a tie-break. Either in //BGP_sanitize_in//, or possibly if we didn't want to inlude peers in //BGP_transit_in//.

Looking at some stats I see we barely use our Lumen transit in Codfw. The reason is that most prefixes we see with the same AS-path length from Arelion and Lumen. But Arelion have MED=0 on the routes they send, and Lumen have various values higher than that, which is getting used as a tie-break, for example: ``` cmooney@re0.cr2-codfw> show route protocol bgp 176.61.33.2 terse inet.0: 937054 destinations, 3246024 routes (936878 active, 61 holddown, 419 hidden) Restart Complete + = Active Route, - = Last Active, * = Both A V Destination P Prf Metric 1 Metric 2 Next hop AS path * V 176.61.0.0/18 B 170 100 0 1299 6830 ? valid >208.80.153.218 208.80.153.213 V B 170 100 22640 3356 6830 ? valid >64.156.73.169 ``` We also had the recent complications from accepting MED inbound from Amazon in eqord and eqiad and how we use it internally. We can wait for the larger evaluation (T297355), but I thought it might be worth discussing either way as it's part of the equation. For my part I don't believe it is serving us well to tie-break on this attribute. My logic being: * We should ultimately control the policy about where we send outbound traffic * Different carriers have different policies, like this between Lumen and Arelion ** Whatever Lumen may be trying to signal, it's definitely not "use Arelion instead" * We should be doing hot-potato routing from our sites towards the internet ** So the only valid MED comparison would be between two routes from the same carrier at the same site I think it would be reasonable, and give us more control plus make things easier to reason about, to just set MED=0 on all inbound routes so it is not used as a tie-break. Either in //BGP_sanitize_in//, or possibly if we didn't want to inlude peers in //BGP_transit_in//.

Looking at some stats I see we barely use our Lumen transit in Codfw. The reason is that most prefixes we see with the same AS-path length from Arelion and Lumen. But Arelion have MED=0 on the routes they send, and Lumen have various values higher than that, which is getting used as a tie-break, for example: ``` cmooney@re0.cr2-codfw> show route protocol bgp 176.61.33.2 terse inet.0: 937054 destinations, 3246024 routes (936878 active, 61 holddown, 419 hidden) Restart Complete + = Active Route, - = Last Active, * = Both A V Destination P Prf Metric 1 Metric 2 Next hop AS path * V 176.61.0.0/18 B 170 100 0 1299 6830 ? valid >208.80.153.218 208.80.153.213 V B 170 100 22640 3356 6830 ? valid >64.156.73.169 ``` We also had the recent complications from accepting MED inbound from Amazon in eqord and eqiad and how we use it internally. We can wait here for the larger evaluation (T297355), but I thought it might be worth discussing here either way as it's part of the equation. FromFor my part I don't believe it is serving us well to tie-break on this attribute. My logic being: * We should ultimately control the policy about where we send outbound traffic * Different carriers have different policies, like this between Lumen and Arelion ** Whatever Lumen may be trying to signal, it's definitely not "use Arelion instead" * We should be doing hot-potato routing from our sites towards the internet ** So the only valid MED comparison would be between two routes from the same carrier at the same site I think it would be reasonable, and give us more control plus make things easier to reason about, to just set MED=0 on all inbound routes so it is not used as a tie-break. Either in //BGP_sanitize_in//, or possibly if we didn't want to inlude peers in //BGP_transit_in//.