To provide fair service to all users of cxserver, we should implement rate limiter to not allow any single user to consume too many resources.
Some considerations for choosing a rate limiter:
- Cxserver is currently stateless, but rate limiting requires state.
- We can either rate limit all type of requests, or just those which consume most resources.
- We can rate limit by the number of requests, or for example amount of characters to translate.
- Rate limiting can be based on IP address or username for services which require authentication.
We can either implement rate limiting our selves, or we could use a existing package such as this or this if redis is available. If we want to implement it our selves, we need to create some kind of system to purge unused state to avoid indefinitely expanding storage as new users use the system. Also need to take care that the state is stored globally and not locally for each thread. Using redis avoid this problem completely, also in case we restart cxserver. The problem with the those two existing packages is that they only allow limiting by request count.