Page MenuHomePhabricator

Can't connect to https://ws-export.wmcloud.org/
Closed, ResolvedPublic

Description

I was able to successfully export a book here
But not here

It hangs for a while, then returns this error:

Wikimedia Cloud Services
Error
This web service cannot be reached. Please contact a maintainer of this project.

Maintainers can find troubleshooting instructions from our documentation on Wikitech.

proxy-03.project-proxy.eqiad1.wikimedia.cloud

I can use the test URL as a workaround, but it seems right to me that 'standard' users should use the 'normal' URL, not the one dedicated to testing...

IMG_1229.png (1×2 px, 146 KB)

Info

iPadOS 17.2 (21C62)
Firefox 134.0 (49061)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Same, I wasn't able to export a book

I've restarted the server, and it appears to be working again. There's no disk space issues (as has been the cause of recent downtime).

The server is still weirdly slow (I already restarted it yesterday) even if it does not seems overloaded (CPU load is around 50%, memory consumption low). Maybe disk disk I/O?

FYI I had to restart it the other day too, and it also was fine on disk space. I've just blocked a few IP ranges that are obvious bots.

There's on range in particular I'm suspicious of that is exporting ~40K books a day. I think it's OK to share a wild card here, but just in case I created a private Paste for @Samwilson and @Tpt to review:

{P75465}

If blocking that IP range keeps the tool functioning then perhaps it's worth it in the short term. It seems like we're just going to keep getting hit by scrapers though. :(

+1 to @Samwilson Blocking an IP is better than having the tool not working for everyone

Having the same issue, a week or two later

I've blocked the range mentioned at P75465 and have restarted Apache. Things are running fine for now, but as Sam says, it probably won't take long before more bots bring it down :(

See also https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/2RFFHXSI6INQDJ2AQ7U3IQ2HTHT5J4VL/

We may want to look into Anubis, which others have reported has worked well for them. I'm going to investigate it for use in XTools as well.

Not working for me on la.wikisource.org currently

Samwilson claimed this task.

I've installed Anubis (in T389713) and the site is back online.