From https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/, "The minimum recommended version of etcd to run in production is 3.2.10+".
Typical, recommended installation is to copy the binary straight from the Github repo, and write some files for it. That's a bit cowboy for WMF, but, as luck would have it, Debian has packaged it's first version at 3.2.18 on Sid!
For building the two clusters we need, we require:
- A backport of the sid/buster package to Stretch in either the WMF repos or Toolforge repos. Because golang, it is likely that the existing package just needs a rebuild with a Stretch target.
- A revisit of the current etcd config in puppet with attention to any new ideas around certificate management. -- note T144153, T169287 and T215553. There's a lot entangled in these tasks.
- Documentation of the build and configuration -- making sure etcdctl actually functions on the servers in a way that is documented and perhaps consistent with upstream docs (currently, it might work, but it is so old that upstream docs are totally different).