ayounsi@bast1003:~$ mtr 216.116.128.28 --report-wide Start: 2021-11-21T07:26:28+0000 HOST: bast1003 Loss% Snt Last Avg Best Wrst StDev 1.|-- ae4-1004.cr2-eqiad.wikimedia.org 0.0% 10 4.5 0.7 0.3 4.5 1.3 2.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.3 0.5 0.3 1.3 0.3 3.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.8 0.8 0.3 3.2 0.9 4.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.3 1.2 0.3 4.7 1.5 5.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.3 1.7 0.3 8.6 2.7 6.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.3 2.6 0.3 8.9 3.1 7.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.3 0.5 0.3 1.7 0.4 8.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.4 0.9 0.4 4.9 1.4 9.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.5 0.7 0.4 2.4 0.6 10.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.5 0.6 0.4 1.5 0.3 11.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.5 0.7 0.4 2.4 0.6 12.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.5 1.2 0.4 4.4 1.3 13.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.5 0.8 0.4 3.0 0.8 14.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 1.4 0.6 0.5 1.4 0.3 15.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.6 2.0 0.5 12.1 3.6 16.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.6 0.8 0.5 2.6 0.7 17.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.5 0.6 0.4 1.0 0.2 18.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.6 0.7 0.5 2.3 0.5 19.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.5 1.3 0.5 5.0 1.5 20.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.6 0.6 0.5 0.7 0.0 21.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.7 1.4 0.5 4.9 1.6 22.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 3.8 1.7 0.6 6.0 1.8 23.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.6 0.7 0.6 1.0 0.1 24.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 1.5 1.0 0.6 1.7 0.4 25.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.7 1.0 0.6 4.4 1.2 26.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 0.7 1.4 0.6 5.3 1.6 27.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 1.4 3.3 0.6 21.4 6.4 28.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 4.8 2.3 0.7 11.2 3.4 29.|-- ae0.cr2-eqiad.wikimedia.org 0.0% 10 0.8 2.0 0.7 11.5 3.4 30.|-- ae0.cr1-eqiad.wikimedia.org 0.0% 10 2.2 2.1 0.7 12.8 3.8 ayounsi@re0.cr1-eqiad> show route 216.116.128.28 extensive inet.0: 860953 destinations, 3772957 routes (860395 active, 26 holddown, 3047 hidden) Restart Complete 216.116.128.0/24 (6 entries, 1 announced) TSI: KRT in-kernel 216.116.128.0/24 -> {indirect(1049116)} unicast reverse-path: 0 [xe-3/3/2.0 xe-4/3/1.0 xe-4/3/2.0 xe-3/0/6.0] Standby generator for 0.0.0.0/0 Page 0 idx 4, (group Confed_eqiad type Internal) Type 1 val 0x7e76ffa8 (adv_entry) Advertised metrics: Flags: Nexthop Change Nexthop: Self MED: 0 Localpref: 250 AS path: [65001] (65020) 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:0 large:24115:1004:6939 Page 1 idx 2, (group Netflow type Internal) Type 1 val 0x7f54f148 (adv_entry) Advertised metrics: Nexthop: 208.115.136.37 MED: 0 Localpref: 250 AS path: [65001] (65020) 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:0 large:24115:1004:6939 Path 216.116.128.0 from 208.80.154.198 Vector len 4. Val: 4 10 *BGP Preference: 170/-251 Next hop type: Indirect, Next hop index: 0 Address: 0x27e45e34 Next-hop reference count: 146499 Source: 208.80.154.198 Next hop type: Router, Next hop index: 0 Next hop: 208.80.154.194 via ae0.0 weight 0x1, selected Session Id: 0x0 Next hop: 208.80.153.221 via xe-4/2/0.0 weight 0xf000 Session Id: 0x0 Protocol next hop: 208.115.136.37 Indirect next hop: 0x7949240 1049116 INH Session ID: 0x2e37a State: Local AS: 65001 Peer AS: 65020 Age: 10:06:58 Metric: 0 Metric2: 243 Validation State: unknown Task: BGP_65020.208.80.154.198+179 Announcement bits (6): 0-KRT 5-Aggregate 7-RT 9-BGP_RT_Background 10-Resolve tree 1 12-Resolve tree 2 AS path: (65020) 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:0 large:24115:1004:6939 Accepted Localpref: 250 Router ID: 208.80.154.198 Indirect next hops: 1 Protocol next hop: 208.115.136.37 Metric: 243 Indirect next hop: 0x7949240 1049116 INH Session ID: 0x2e37a Indirect path forwarding next hops: 2 Next hop type: Router Next hop: 208.80.154.194 via ae0.0 weight 0x1 Session Id: 0x0 Next hop: 208.80.153.221 via xe-4/2/0.0 weight 0xf000 Session Id: 0x0 208.115.136.0/23 Originating RIB: inet.0 Metric: 243 Node path count: 1 Forwarding nexthops: 2 Nexthop: 208.80.154.194 via ae0.0 Session Id: 0 Nexthop: 208.80.153.221 via xe-4/2/0.0 Session Id: 0 BGP Preference: 170/-251 Next hop type: Router, Next hop index: 2440 Address: 0xaea9c1c Next-hop reference count: 172672 Source: 206.126.239.251 Next hop: 206.126.236.37 via xe-3/0/6.0, selected Session Id: 0x30f17 State: Inactive reason: Not Best in its group - Route Metric or MED comparison Local AS: 65001 Peer AS: 24115 Age: 5:49 Metric: 15 Validation State: unknown Task: BGP_24115.206.126.239.251+179 AS path: 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:6 large:24115:1004:6939 Accepted Localpref: 250 Router ID: 206.126.239.251 Merit (last update/now): 500/384 damping-parameters: default Last update: 00000000:05:49 First update: 00000000:05:49 Flaps: 1 BGP Preference: 170/-251 Next hop type: Router, Next hop index: 2440 Address: 0xaea9c1c Next-hop reference count: 172672 Source: 206.126.239.252 Next hop: 206.126.236.37 via xe-3/0/6.0, selected Session Id: 0x30f17 State: Inactive reason: Not Best in its group - Route Metric or MED comparison Local AS: 65001 Peer AS: 24115 Age: 1:05:21 Metric: 15 Validation State: unknown Task: BGP_24115.206.126.239.252+179 AS path: 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:2 large:24115:1003:0 large:24115:1004:6939 Accepted Localpref: 250 Router ID: 206.126.239.252 BGP Preference: 170/-101 Next hop type: Router, Next hop index: 1208 Address: 0x4cde60c Next-hop reference count: 155258 Source: 216.66.30.89 Next hop: 216.66.30.89 via xe-4/3/1.0, selected Session Id: 0x45c State: Inactive reason: Local Preference Local AS: 65001 Peer AS: 6939 Age: 2d 15:37:13 Metric: 0 Validation State: unknown Task: BGP_6939.216.66.30.89+8084 AS path: 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 6939:1000 6939:7152 6939:8840 6939:9001 14907:4 unknown iana opaque 0x4300:0x0:0x1 Accepted Localpref: 100 Router ID: 216.218.252.169 BGP Preference: 170/-101 Next hop type: Router, Next hop index: 2144 Address: 0x4d0e06c Next-hop reference count: 906647 Source: 4.16.71.245 Next hop: 4.16.71.245 via xe-3/3/2.0, selected Session Id: 0x11692 State: Inactive reason: Local Preference Local AS: 65001 Peer AS: 3356 Age: 1d 22:03:25 Metric: 20020 Validation State: unknown Task: BGP_3356.4.16.71.245+61848 AS path: 3356 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 3356:3 3356:22 3356:86 3356:575 3356:666 3356:903 3356:2057 14907:4 unknown iana opaque 0x4300:0x0:0x1 Accepted Localpref: 100 Router ID: 4.69.204.33 BGP Preference: 170/-51 Next hop type: Router, Next hop index: 1157 Address: 0x933ab3c Next-hop reference count: 965874 Source: 192.80.17.185 Next hop: 192.80.17.185 via xe-4/3/2.0, selected Session Id: 0x45f State: Inactive reason: Local Preference Local AS: 65001 Peer AS: 2914 Age: 1d 21:11:55 Metric: 2001 Validation State: unknown Task: BGP_2914.192.80.17.185+60441 AS path: 2914 174 12183 ? Aggregator: 12183 216.116.128.1 Communities: 2914:420 2914:1001 2914:2000 2914:3000 14907:0 14907:4 unknown iana opaque 0x4300:0x0:0x1 Accepted Localpref: 50 Router ID: 129.250.0.219 ayounsi@re0.cr2-eqiad> show route 216.116.128.28 extensive inet.0: 862997 destinations, 2831592 routes (860385 active, 1 holddown, 4045 hidden) Restart Complete 216.116.128.0/24 (5 entries, 1 announced) TSI: KRT in-kernel 216.116.128.0/24 -> {indirect(1048680)} unicast reverse-path: 0 [ae5.0 xe-3/3/3.0] Standby generator for 0.0.0.0/0 Page 0 idx 7, (group Netflow type Internal) Type 1 val 0xa15ca638 (adv_entry) Advertised metrics: Nexthop: 208.80.154.196 MED: 0 Localpref: 250 AS path: [65001] (65020) 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:0 large:24115:1004:6939 Cluster ID: 208.80.154.197 Originator ID: 208.80.154.196 Path 216.116.128.0 from 208.80.154.196 Vector len 4. Val: 7 *BGP Preference: 170/-251 Next hop type: Indirect, Next hop index: 0 Address: 0x4ceca0c Next-hop reference count: 493265 Source: 208.80.154.196 Next hop type: Router, Next hop index: 0 Next hop: 208.80.154.193 via ae0.0 weight 0x1, selected Session Id: 0x0 Next hop: 208.80.154.207 via ae1.402 weight 0xf000 Session Id: 0x0 Protocol next hop: 208.80.154.196 Indirect next hop: 0x7905c00 1048680 INH Session ID: 0x1136f State: Local AS: 65001 Peer AS: 65001 Age: 10:07:16 Metric: 0 Metric2: 3 Validation State: unknown Task: BGP_65001.208.80.154.196+56195 Announcement bits (6): 0-KRT 6-Aggregate 8-RT 9-BGP_RT_Background 10-Resolve tree 1 12-Resolve tree 2 AS path: (65020) 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:0 large:24115:1004:6939 Accepted Localpref: 250 Router ID: 208.80.154.196 Indirect next hops: 1 Protocol next hop: 208.80.154.196 Metric: 3 Indirect next hop: 0x7905c00 1048680 INH Session ID: 0x1136f Indirect path forwarding next hops: 2 Next hop type: Router Next hop: 208.80.154.193 via ae0.0 weight 0x1 Session Id: 0x0 Next hop: 208.80.154.207 via ae1.402 weight 0xf000 Session Id: 0x0 208.80.154.196/32 Originating RIB: inet.0 Metric: 3 Node path count: 1 Forwarding nexthops: 2 Nexthop: 208.80.154.193 via ae0.0 Session Id: 0 Nexthop: 208.80.154.207 via ae1.402 Session Id: 0 BGP Preference: 170/-251 Next hop type: Indirect, Next hop index: 0 Address: 0xbc4f782c Next-hop reference count: 148132 Source: 208.80.154.198 Next hop type: Router, Next hop index: 0 Next hop: 208.80.154.209 via xe-4/2/0.0 weight 0x1, selected Session Id: 0x0 Next hop: 208.80.154.215 via xe-3/2/2.0 weight 0xf000 Session Id: 0x0 Protocol next hop: 208.115.136.37 Indirect next hop: 0x7930a00 1048795 INH Session ID: 0xfdae State: Inactive reason: Not Best in its group - IGP metric Local AS: 65001 Peer AS: 65020 Age: 10:07:16 Metric: 0 Metric2: 240 Validation State: unknown Task: BGP_65020.208.80.154.198+179 AS path: (65020) 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:0 large:24115:1004:6939 Accepted Localpref: 250 Router ID: 208.80.154.198 Indirect next hops: 1 Protocol next hop: 208.115.136.37 Metric: 240 Indirect next hop: 0x7930a00 1048795 INH Session ID: 0xfdae Indirect path forwarding next hops: 2 Next hop type: Router Next hop: 208.80.154.209 via xe-4/2/0.0 weight 0x1 Session Id: 0x0 Next hop: 208.80.154.215 via xe-3/2/2.0 weight 0xf000 Session Id: 0x0 208.115.136.0/23 Originating RIB: inet.0 Metric: 240 Node path count: 1 Forwarding nexthops: 2 Nexthop: 208.80.154.209 via xe-4/2/0.0 Session Id: 0 Nexthop: 208.80.154.215 via xe-3/2/2.0 Session Id: 0 BGP Preference: 170/-251 Next hop type: Router, Next hop index: 916 Address: 0x3277c8dc Next-hop reference count: 172796 Source: 206.126.239.251 Next hop: 206.126.236.37 via xe-3/3/3.0, selected Session Id: 0x102c3 State: Inactive reason: Not Best in its group - Route Metric or MED comparison Local AS: 65001 Peer AS: 24115 Age: 6:06 Metric: 15 Validation State: unknown Task: BGP_24115.206.126.239.251+179 AS path: 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:6 large:24115:1004:6939 Accepted Localpref: 250 Router ID: 206.126.239.251 Merit (last update/now): 500/378 damping-parameters: default Last update: 00000000:06:06 First update: 00000000:06:06 Flaps: 1 BGP Preference: 170/-251 Next hop type: Router, Next hop index: 916 Address: 0x3277c8dc Next-hop reference count: 172796 Source: 206.126.239.252 Next hop: 206.126.236.37 via xe-3/3/3.0, selected Session Id: 0x102c3 State: Inactive reason: Not Best in its group - Route Metric or MED comparison Local AS: 65001 Peer AS: 24115 Age: 1:05:38 Metric: 15 Validation State: unknown Task: BGP_24115.206.126.239.252+179 AS path: 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:2 large:24115:1003:0 large:24115:1004:6939 Accepted Localpref: 250 Router ID: 206.126.239.252 BGP Preference: 170/-101 Next hop type: Router, Next hop index: 2477 Address: 0x4d1c0dc Next-hop reference count: 1128604 Source: 80.239.132.225 Next hop: 80.239.132.225 via ae5.0, selected Session Id: 0x41fa State: Inactive reason: Local Preference Local AS: 65001 Peer AS: 1299 Age: 2d 15:37:29 Validation State: unknown Task: BGP_1299.80.239.132.225+62380 AS path: 1299 6939 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 1299:35000 14907:4 unknown iana opaque 0x4300:0x0:0x1 Accepted Localpref: 100 Router ID: 2.255.254.62 ========================================== cr1-eqord Source: 208.80.154.198 -< cr2-eq Next hop: 208.80.154.194 via ae0.0 weight 0x1, selected Session Id: 0x0 Next hop: 208.80.153.221 via xe-4/2/0.0 weight 0xf000 Session Id: 0x0 Protocol next hop: 208.115.136.37 Indirect next hop: 0x7949240 1049116 INH Session ID: 0x2e37a ========================================= ayounsi@re0.cr1-eqiad> show route 216.116.128.28 receive-protocol bgp 208.80.154.198 extensive inet.0: 860918 destinations, 3772889 routes (860386 active, 0 holddown, 2928 hidden) Restart Complete * 216.116.128.0/24 (6 entries, 1 announced) Accepted Nexthop: 208.115.136.37 <- in eqord MED: 0 Localpref: 250 AS path: (65020) 6939 12183 ? Aggregator: 12183 216.116.128.1 Communities: 14907:3 24115:6939 24115:24115 24115:65023 unknown iana opaque 0x4300:0x0:0x1 large:24115:1000:2 large:24115:1001:1 large:24115:1002:1 large:24115:1003:0 large:24115:1004:6939 ayounsi@re0.cr1-eqiad> show route 208.115.136.37 extensive inet.0: 860978 destinations, 3773111 routes (860445 active, 1 holddown, 2671 hidden) Restart Complete 208.115.136.0/23 (1 entry, 1 announced) TSI: KRT in-kernel 208.115.136.0/23 -> {list:208.80.154.194, 208.80.153.221} Standby generator for 0.0.0.0/0 *OSPF Preference: 150 Next hop type: Router, Next hop index: 0 Address: 0x4e8e8bc Next-hop reference count: 407 Next hop: 208.80.154.194 via ae0.0 weight 0x1, selected Session Id: 0x0 Next hop: 208.80.153.221 via xe-4/2/0.0 weight 0xf000 Session Id: 0x0 State: Local AS: 65001 Age: 3d 14:09:49 Metric: 243 Validation State: unverified Tag: 0 Task: OSPF Announcement bits (6): 0-KRT 5-Aggregate 7-RT 10-Resolve tree 1 11-Resolve_IGP_FRR task 12-Resolve tree 2 AS path: I After disabling Confed_eqord on cr1-eqiad: ayounsi@re0.cr2-eqiad> show route 216.116.128.28 inet.0: 862999 destinations, 2734556 routes (860429 active, 1 holddown, 3673 hidden) Restart Complete + = Active Route, - = Last Active, * = Both 216.116.128.0/24 *[BGP/170] 00:01:44, MED 0, localpref 250, from 208.80.154.196 AS path: 6939 12183 ?, validation-state: unknown > to 208.80.154.193 via ae0.0 to 208.80.154.207 via ae1.402 [BGP/170] 10:26:52, MED 0, localpref 250, from 208.80.154.198 AS path: (65020) 6939 12183 ?, validation-state: unknown > to 208.80.154.209 via xe-4/2/0.0 to 208.80.154.215 via xe-3/2/2.0 [BGP/170] 00:25:42, MED 15, localpref 250, from 206.126.239.251 AS path: 6939 12183 ?, validation-state: unknown > to 206.126.236.37 via xe-3/3/3.0 [BGP/170] 01:25:14, MED 15, localpref 250, from 206.126.239.252 AS path: 6939 12183 ?, validation-state: unknown > to 206.126.236.37 via xe-3/3/3.0 [BGP/170] 2d 15:57:05, localpref 100 AS path: 1299 6939 6939 12183 ?, validation-state: unknown > to 80.239.132.225 via ae5.0