A user on IRC reported that it's not possible to connect to Wikimedia websites since approximately 15 hours ago through this ISP. Apparently, other users of this ISP are also unable to connect. IP range affected is 41.151.0.0/16.
Description
Event Timeline
@Glaisher: If the user, or another one experiencing the same issue are still on IRC, could you please ask them to complete a traceroute and attach it to the ticket to assist the ops them?
Hi everybody, and thanks for all the help!
Here is the ping:
Pinging en.wikipedia.org [91.198.174.192] with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Ping statistics for 91.198.174.192:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
So it resolves the IP fine but nothing gets through
Here are the details of the tracert:
Tracing route to en.wikipedia.org [91.198.174.192]
over a maximum of 30 hops:
1 * * * Request timed out. 2 38 ms 49 ms 39 ms 172.17.32.13 3 68 ms 69 ms 69 ms 172.17.32.21 4 77 ms 91 ms 68 ms 8ta-150-177-01.telkomadsl.co.za [41.150.177.1] 5 56 ms 69 ms 69 ms 196.25.63.233 6 208 ms 199 ms 199 ms 196.43.9.86 7 207 ms 210 ms 209 ms tenge7-2.var01.ldn04.pccwbtn.net [63.218.243.17] 8 217 ms 219 ms 229 ms ae2.cr2-esams.wikimedia.org [80.249.209.176] 9 * * * Request timed out.
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 * * * Request timed out.
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.
Trace complete.
If you need any other information, let me know. I am currently connecting to Wikpedia, Wikimedia and Phabricator via an IP hiding service based in the USA, but it will expire in three days, so I need to find a solution before then. My best guess is that our IP range has been blocked due to hacking attempts.
My own IP address is currently 41.144.175.* so I think the IP range might be larger than the one officially published. I'm trying to track down a service which will reveal their full IP range, but so far no luck. However, I did find this:
Telkom is announcing their routes directly to us over AMS-IX via the route servers, not a direct peering. However, it seems that they're blackholing all the traffic we're sending them right over from the first hop (on traceroute not even the first hop is appearing).
I'll contact them about it, but in the meantime I've downprefed the direct route and going via transits now.
@Cadar, could you retry and let us know if things work now? Thanks!
Many thanks for the help! I have disabled my IP hider and seem to be able to connect fine now. I've also asked a couple of other people to check that they can connect properly as well. It doesn't surprise me that Telkom have managed to break Wikipedia, it's pretty much typical for them. The Symantec security certificate on their site expired at the end of last month and it turned out they had updated the site with new subdomains and neglected to reroute the traffic from the old pages. That's the joy of using what are laughingly called "service providers" in the Third World for you.
New tracert results:
Tracing route to wikipedia.org [91.198.174.192]
over a maximum of 30 hops:
1 * * * Request timed out. 2 38 ms 29 ms 39 ms 172.17.32.9 3 62 ms 69 ms 69 ms 172.17.32.17 4 58 ms 69 ms 69 ms 8ta-150-177-02.telkomadsl.co.za [41.150.177.2] 5 76 ms 69 ms 90 ms 196.25.63.234 6 66 ms 69 ms 69 ms 196.25.7.121 7 197 ms 189 ms 199 ms lon-ip-hsll-1-gig-0-1-0.telkom-ipnet.co.za [196.43.9.46] 8 210 ms 239 ms 199 ms tenge7-2.var01.ldn04.pccwbtn.net [63.218.243.17] 9 * * * Request timed out.
10 217 ms 239 ms 229 ms text-lb.esams.wikimedia.org [91.198.174.192]
Trace complete.
So we're back in business! Thanks Faidon, you just saved most of Southern Africa from a long dark time of Wikipedia withdrawals 8-)
Telkom did not respond despite my repeated emails, but I've just removed the explicit route preference and traceroutes seem to indicate that traffic to 41.144.175.0/24 still works.
@Cadar, if connectivity is broken for you now, please reopen this task — and, of course, if you notice any more troubles, please feel free to open a new task. Thanks!
Hi Faidon, thanks for the support on the access question. We seem to be up
and connected to Wikipedia fine on this side. I've noticed similar issues
with some other sites in the last week or two. Perhaps it was just system
upgrades or something. It's typical that they would just ignore requests
for help like that though.
Cheers,
Jon