Since version v1.5.13, memcached supports TLS!
**Problem**
Backstory: Our mcrouter instances have 2 server pools: one that includes all mc* hosts on the local primary DC and another pool that consists of 4 mw* servers which act as a "mcrouter proxy" to the other primary DC, for example in eqiad we have:
```
"codfw": {
"servers": [
"10.192.0.61:11214:ascii:ssl",
"10.192.16.56:11214:ascii:ssl",
"10.192.32.113:11214:ascii:ssl",
"10.192.48.94:11214:ascii:ssl"
]
},
"eqiad": {
"servers": [
"10.64.0.80:11211:ascii:plain",
"10.64.0.81:11211:ascii:plain",
"10.64.0.82:11211:ascii:plain",
"10.64.0.83:11211:ascii:plain",
"10.64.0.84:11211:ascii:plain",
"10.64.16.107:11211:ascii:plain",
"10.64.16.108:11211:ascii:plain",
"10.64.16.109:11211:ascii:plain",
"10.64.16.110:11211:ascii:plain",
"10.64.32.208:11211:ascii:plain",
"10.64.32.209:11211:ascii:plain",
"10.64.32.210:11211:ascii:plain",
"10.64.32.211:11211:ascii:plain",
"10.64.32.212:11211:ascii:plain",
"10.64.48.155:11211:ascii:plain",
"10.64.48.156:11211:ascii:plain",
"10.64.48.157:11211:ascii:plain",
"10.64.48.158:11211:ascii:plain"
]
}
```
**Goal**
If we were to enable TLS, will eliminate the need to use those "mcrouter proxies", and secure connectivity between mediawiki and the memcached cluster. This will eliminate 4 snowflake mediawiki servers from production! We can run memcached on two ports, a TLS one, for cross-dc replication, and a nontls one for local datacentre traffic.
**Versions:**
* v1.6.6: we have this version packaged and ready, but it will need to be deployed with caution since there are some changes which can affect a busy cluster like ours
**How? (mediawiki is on eqiad)**
[] Create the relevant puppet changes and add the `dual_listener` feature flag
[] Test on mwdebug2001: we can enable "dual_listener" on mc2019, add it on its mcrouter pool and run a simple url list against mwdebug2001.
[] Given the number and the size of keys we replicate is very small, we can replace 1 mcrouter proxy with an actual memcached server on codfw
[] We can then start adding all codfw servers either one at a time, or the whole cluster at once (TBD)
**Notes**
We could consider switching all memcached traffic to TLS, but this comes with a major drawback: all tools that can provide real time key traffic (such as memkeys etc all), practically dump the network traffic. If this traffic is encrypted, the tools become useless. We are going to solve this problem at a later time.