We strongly need a consistent key-value store to use in cluster discovery/coordination and configuration.
Potential candidates with substantial adoption, development community health, and feature-completeness are at the moment in my opinion:
- etcd
- consul
- Zookeeper
The chosen product should at least:
* Guarantee read-availability during a node failure, and consistency and recovery after a failure to be easily understandable and manageable
* have a decent write performance, but an extremely good read performance with very low latencies in all operating conditions
* Allow clients to watch the value of a key/ a tree for changes
* Allow easy backups
* Work cross-datacenter (even with some limitations)
* Have clean client libraries in most languages we use at the WMF
* Allow (force?) encrypted connections from clients
Bonuses:
* Easy to query from the CLI
* Allow some level of authentication/grants system
* Packaged in debian