Page MenuHomePhabricator

Set up backend per-IP limits on varnish for WDQS
Closed, ResolvedPublic

Description

We need to create a backend per-IP limit on varnish for WDQS, that would allow only set number of requests per IP (say, 10) to be sent to the backend in parallel and the rest of the requests should be queued by varnish and expired if the are queued for too long, or just rejected outright if that's easier. This will ensure no client would hog all available backend time.

Since we have Trusted XFF support, we probably need to use that too when resolving IPs.

Event Timeline

Smalyshev raised the priority of this task from to Medium.
Smalyshev updated the task description. (Show Details)
Smalyshev subscribed.

It would be best to use the header X-Client-IP as the notion of the client IP address for these sorts of purposes. This is intended to resolve trusted XFF, but has a much shorter list (intended to be improved on), whereas TrustedXFF is not usable by the the varnish layer and has a much longer list, but the list is not well-maintained.

We don't have any mechanism for giving an explicit parallelism limit per-client-IP (which only makes sense for requests that commonly run for long times anyways). What we do have at our disposal currently is ratelimiting per-client-IP, and we could even modify the weighting based on response headers as well.

Smalyshev claimed this task.

I think this is done.