This morning, we got an alert that mr1-eqiad.oob IPv6 is down
Description
Description
Event Timeline
Comment Actions
Thanks!
It's back, I'd guess a transient error on Equinix's network. Not worth investigating it more as it's now up and only OOB.
Comment Actions
[19:41] <icinga-wm> PROBLEM - Host mr1-eqiad.oob IPv6 is DOWN: PING CRITICAL - Packet loss = 100%
Comment Actions
not working
ayounsi@icinga1001:~$ mtr -z --report-wide 2607:f6f0:205::153 Start: Tue Jan 28 19:51:46 2020 HOST: icinga1001 Loss% Snt Last Avg Best Wrst StDev 1. AS14907 ae3-1003.cr1-eqiad.wikimedia.org 0.0% 10 0.3 0.5 0.3 1.3 0.0 2. AS2914 xe-0-0-28-0.a03.asbnva02.us.bb.gin.ntt.net 0.0% 10 0.5 4.3 0.3 20.0 7.9 3. AS2914 ae-70.r06.asbnva02.us.bb.gin.ntt.net 0.0% 10 1.3 1.0 0.9 1.3 0.0 4. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 ayounsi@icinga1001:~$
working
ayounsi@bast4002:~$ mtr -z --report-wide 2607:f6f0:205::153 Start: Tue Jan 28 19:51:47 2020 HOST: bast4002 Loss% Snt Last Avg Best Wrst StDev 1. AS14907 et-1-0-1-1201.cr3-ulsfo.wikimedia.org 0.0% 10 0.3 3.5 0.2 30.4 9.4 2. AS14907 et-0-0-0-2.cr4-ulsfo.wikimedia.org 0.0% 10 0.5 0.7 0.2 4.0 1.1 3. AS2914 xe-0-1-0-3-6.r05.plalca01.us.bb.gin.ntt.net 0.0% 10 1.3 1.4 1.3 1.6 0.0 4. AS2914 ae-15.r01.snjsca04.us.bb.gin.ntt.net 0.0% 10 2.1 2.0 2.0 2.1 0.0 5. AS2914 ae-1.r22.snjsca04.us.bb.gin.ntt.net 0.0% 10 1.9 1.9 1.8 2.2 0.0 6. AS2914 ae-7.r23.asbnva02.us.bb.gin.ntt.net 0.0% 10 64.0 64.3 64.0 65.0 0.0 7. AS2914 ae-2.r05.asbnva02.us.bb.gin.ntt.net 0.0% 10 68.5 68.5 68.4 68.6 0.0 8. AS2914 ae-0.a02.asbnva02.us.bb.gin.ntt.net 0.0% 10 69.8 71.1 67.8 78.3 3.4 9. AS2914 2001:418:0:5000::aab 0.0% 10 61.5 62.4 61.4 70.4 2.8 10. AS12085 2607:f6f0:1000:1af::2 0.0% 10 61.9 64.8 61.8 90.9 9.2 11. AS12085 ge-0-0-5.mr1-eqiad.wikimedia.org 0.0% 10 63.1 64.1 63.0 72.6 2.9
Something seems off either within NTT or with the NTT/Equinix hand-off
Emailed NTT.