Page MenuHomePhabricator

Request to enable XFF headers for wikisource VPS project
Closed, ResolvedPublic

Description

Project Name:
wikisource

Request:
Enable XFF headers for the wikisource VPS project, or if it is done on a per-instance level, specifically wsexport-prod01.wikisource.eqiad.wmflabs

Rationale:
Similar to the situation with XTools (see T135046#6056334), WS Export is subject to significant disruptive automation by web crawlers and click-happy bots, which can compromise the health and uptime of our service. Community-Tech maintains the tool and have blocked most everything we can by user agent, but some users (which I can only suspect are bots, but nonetheless are disruptive) continue to hammer away, very often with human-like user agents that we can't block without risk of collateral damage.

Having the ability to block IPs in XTools has been absolute blessing. Stability has been better than ever as a result. We are requesting to have the same luxury with WS Export so that we can ensure our users have a dependable service. Thank you for the consideration!

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I'll note all wikisource project members are staff, apart from one volunteer who may or may have not signed any NDAs. They are a highly trusted user and I'm sure we can arrange they sign any applicable documents if necessary.

Andrew subscribed.

As long as this is limited to that particular project this seems fine.

Andrew triaged this task as Medium priority.Apr 13 2021, 4:13 PM
Andrew moved this task from Inbox to Soon! on the cloud-services-team (Kanban) board.

Done! I also enabled XFF for wsexport.wmcloud.org in case you migrate to the new domain name.

Done! I also enabled XFF for wsexport.wmcloud.org in case you migrate to the new domain name.

Thanks! We're actually at https://ws-export.wmcloud.org these days (note the hyphen-minus -), so I don't think your commit had any effect. I should have provided the URL, my apologies.

Mentioned in SAL (#wikimedia-cloud) [2021-04-15T09:15:05Z] <arturo> refresh hiera XFF entry for ws-export.wmcloud.org T279111

@aborrero Is there any chance you could add https://ws-export-test.wmcloud.org/ too? We have some new throttling logic we'd like to test outside our production instance. Thanks!

Mentioned in SAL (#wikimedia-cloud) [2022-02-02T17:05:03Z] <arturo> add ws-export-test.wmcloud.org to XFF allowlist (af9e47341ad063f3f0c34b516f8963f7fe82cff9) (T279111)