I looked a little bit into ringpop today. It only supports node <= 0.10 currently, and even when using that (via nvm) the examples don't actually work.

The main issue for 4.2 compatibility is a relatively long list of incompatible binary dependencies:

• https://github.com/lloyd/node-toobusy
• https://github.com/lovell/farmhash
• https://github.com/uber/tchannel-node

I think it's worth looking for alternatives before investing more time into ringpop.

https://github.com/kadtools/kad and related tools look promising. This is a simple extensible DHT framework, with UDP transports and some layered functionality like the pub-sub. The examples work out of the box, and a custom decaying counter key-value store looks rather simple to implement based on this. But, we'd need to invest a bit of time to optimize this for throughput and latency.

A basic rate limiter is now available at https://github.com/gwicke/limitation. Performance is decent, with thousands of keys, 15 DHT nodes and ~100k req/s handled in a single process.

I'm proposing to integrate this into service-runner with the following strategy:

• Set up a shared rate limiter instance in the master node, using the default port 3050 or a random fall-back port (already handled in kad-ratelimiter).
• Have a copy of counters and blocks in each worker, and perform all checks / increments in-process.
• Periodically, push counters to the master process. The master process periodically updates with DHT & sends updated blocks back to workers.

Connecting only the master process to the DHT avoids an excessive number of DHT nodes, and side-steps the issues around cluster. Based on testing, stability and load should not be an issue.

PR available at https://github.com/wikimedia/service-runner/pull/89, plus a small one for hyperswitch: https://github.com/wikimedia/hyperswitch/pull/12 to forward the ratelimiter property.

@Pchelolo found this useful listing of rate limiting related headers on stackoverflow. The "-remaining" headers in particular make it relatively easy for clients to pace themselves.

However, implementing these without introducing synchronous check overheads looks a bit tricky. While each limitation instance has a full set of current counter values available for all keys it has received requests for, there are challenges from

• the delay inherent in asynchronous counter exchanges, and
• incomplete counter sets for very low-volume entry points.

It might be possible to still solve this with heuristics, but this will need some more thinking.