Page MenuHomePhabricator

Prioritize underdog IXP
Closed, ResolvedPublic

Description

For sites where we peer at multiple IXPs, for example ulsfo where we have Equinix and SFMIX, we want to be able to prioritize one over the other.

  • Better latency (SFMIX is closer in term of ms)
  • Better balancing (SFMIX is smaller and thus less used)

As a side note I documented all the BGP communities we have defined in https://wikitech.wikimedia.org/wiki/IP_and_AS_allocations#BGP_communities

I introduced a new BGP community PEERING_ROUTE_PRIMARY which set the local-pref of 10 above the regular PEERING_ROUTE.

As well as a new policy BGP_IXP_PRIMARY_in duplicating BGP_IXP_in to be applied to the matching groups (eg. IX4/6).

To be configured in Homer of course.

[edit policy-options]
+   policy-statement BGP_IXP_PRIMARY_in {
+       term filter {
+           from as-path-group AS-PATH-FILTER;
+           then reject;
+       }
+       term avoid-paths {
+           from as-path-group AVOID-PATHS;
+           then {
+               community add AVOIDED_PATH;
+           }
+       }
+       term rpki-classification {
+           from policy BGP_rpki;
+       }
+       term selected-paths {
+           from as-path-group SELECTED-PATHS;
+           then {
+               community add SELECTED_PATH;
+           }
+       }
+       term damping {
+           then damping default;
+       }
+       then {
+           community add PEERING_ROUTE_PRIMARY;
+           next policy;
+       }
+   }
[edit policy-options policy-statement BGP_community_actions]
     term ixp-peer { ... }
+    term ixp-primary-peer {
+        from community PEERING_ROUTE_PRIMARY;
+        then {
+            local-preference 260;
+            next policy;
+        }
+    }
[edit policy-options]
+   community PEERING_ROUTE_PRIMARY members 14907:2;
[edit protocols bgp group IX4]
-    import [ BGP_sanitize_in BGP_IXP_in BGP_community_actions ];
+    import [ BGP_sanitize_in BGP_IXP_PRIMARY_in BGP_community_actions ];
[edit protocols bgp group IX4 neighbor 206.197.187.253]
-     import [ BGP_sanitize_in BGP_IXP_RS_in BGP_IXP_in BGP_community_actions ];
+     import [ BGP_sanitize_in BGP_IXP_RS_in BGP_IXP_PRIMARY_in BGP_community_actions ];
[edit protocols bgp group IX4 neighbor 206.197.187.254]
-     import [ BGP_sanitize_in BGP_IXP_RS_in BGP_IXP_in BGP_community_actions ];
+     import [ BGP_sanitize_in BGP_IXP_RS_in BGP_IXP_PRIMARY_in BGP_community_actions ];
[edit protocols bgp group IX6]
-    import [ BGP_sanitize_in BGP_IXP_in BGP_community_actions ];
+    import [ BGP_sanitize_in BGP_IXP_PRIMARY_in BGP_community_actions ];
[edit protocols bgp group IX6 neighbor 2001:504:30::ba06:3055:1]
-     import [ BGP_sanitize_in BGP_IXP_RS_in BGP_IXP_in BGP_community_actions ];
+     import [ BGP_sanitize_in BGP_IXP_RS_in BGP_IXP_PRIMARY_in BGP_community_actions ];
[edit protocols bgp group IX6 neighbor 2001:504:30::ba06:3055:2]
-     import [ BGP_sanitize_in BGP_IXP_RS_in BGP_IXP_in BGP_community_actions ];
+     import [ BGP_sanitize_in BGP_IXP_RS_in BGP_IXP_PRIMARY_in BGP_community_actions ];

Event Timeline

ayounsi triaged this task as Medium priority.Sep 10 2020, 8:56 AM
ayounsi created this task.
Restricted Application added a project: Operations. · View Herald TranscriptSep 10 2020, 8:56 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 629056 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/homer/public@master] Add policy-options for primary IXPs

https://gerrit.wikimedia.org/r/629056

Mentioned in SAL (#wikimedia-operations) [2020-09-22T09:22:22Z] <XioNoX> add BGP_IXP_PRIMARY_in to cr3-ulsfo - T262517

Mentioned in SAL (#wikimedia-operations) [2020-09-22T09:24:27Z] <XioNoX> replace BGP_IXP_in with BGP_IXP_PRIMARY_in on cr3-ulsfo IX BGP group - T262517

Change 629056 merged by Ayounsi:
[operations/homer/public@master] Add policy-options for primary IXPs

https://gerrit.wikimedia.org/r/629056

Mentioned in SAL (#wikimedia-operations) [2020-09-22T10:51:54Z] <XioNoX> Add policy-options for primary IXPs to all routers - T262517

Change 629100 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/homer/public@master] Introduce primary_ixp variable

https://gerrit.wikimedia.org/r/629100

Change 629100 merged by jenkins-bot:
[operations/homer/public@master] Introduce primary_ixp variable

https://gerrit.wikimedia.org/r/629100

ayounsi closed this task as Resolved.Tue, Sep 22, 12:24 PM

All done in ulsfo, other sites to follow when they are turned on.

CDanis added a subscriber: CDanis.Tue, Sep 22, 12:58 PM