Page MenuHomePhabricator

cxserver: rate limiting
Closed, ResolvedPublic

Description

To provide fair service to all users of cxserver, we should implement rate limiter to not allow any single user to consume too many resources.

Some considerations for choosing a rate limiter:

  • Cxserver is currently stateless, but rate limiting requires state.
  • We can either rate limit all type of requests, or just those which consume most resources.
  • We can rate limit by the number of requests, or for example amount of characters to translate.
  • Rate limiting can be based on IP address or username for services which require authentication.

We can either implement rate limiting our selves, or we could use a existing package such as this or this if redis is available. If we want to implement it our selves, we need to create some kind of system to purge unused state to avoid indefinitely expanding storage as new users use the system. Also need to take care that the state is stored globally and not locally for each thread. Using redis avoid this problem completely, also in case we restart cxserver. The problem with the those two existing packages is that they only allow limiting by request count.

Event Timeline

Nikerabbit claimed this task.
Nikerabbit raised the priority of this task from to High.
Nikerabbit updated the task description. (Show Details)
Nikerabbit added subscribers: Arrbee, KartikMistry, santhosh and 2 others.

@GWicke, Could you please advice what is the best option here? We need this kind of gating to use MT systems that are hosted outside our cluster.

santhosh edited a custom field.

@santhosh, rate limiting is indeed tricky and something we need for our APIs in general. To clarify, are the requests you are interested in external (from public IPs) or internal?

Could be both, but currently the requests for MT on Special:ContentTranslation come directly from browser to cxserver.

[ Meta: Not sure how to classify it in the workboard. Bugs? Tech debt? Create a new column for Performance? Something else? @Arrbee, @Nikerabbit - suggestions are welcome. ]

Is there any update about this, now that we moved to service runner?

We did look into rate limiting options using redis, but would prefer to avoid the complexity & SPOF issues around that.

I thought we already had a tracking bug for rate limiting, but couldn't find it. So, I created T125123 to track progress.

@Amire80, @Nikerabbit: Rate limiter functionality is now part of service-runner, and is also about to be used in RESTBase. See the module documentation.

As a quick update, we have been using the service-runner rate limiting functionality in production for several weeks now, and have not encountered any issues.

GWicke lowered the priority of this task from High to Medium.Oct 12 2016, 10:28 PM
GWicke edited projects, added Services (watching); removed Services.
santhosh claimed this task.