===Why?
Thumbor is using Memcached via Nutcracker. Right now, Thumbor's Memcached backend is a group of small Memcached instances running on each physical Thumbor host, accessed (and sharded) via Nutcracker. Nutcraker is a software we would like to completely retire from our infrastructure, while, with Thumbor's migration to K8s, the physical servers will be going away.
* about ~1 MB allocated on each host https://grafana-rw.wikimedia.org/d/000000316/memcache?orgId=1&var-datasource=eqiad%20prometheus%2Fops&var-cluster=thumbor&var-instance=All
* ~40 objects stored in each server https://grafana-rw.wikimedia.org/d/000000317/memcache-slabs?orgId=1&var-datasource=eqiad%20prometheus%2Fops&var-cluster=thumbor&var-instance=thumbor1005&var-slab=All
===What?
While we are still working on the details as to how Mcrouter will exist in kubernetes, we can proceed in liberating the bare metal thumbor servers from the weight of having a memcached instance. A rational solution would be to create 2 VMs in each DC. One server to be the main mecached host for thumbor, and the second to be the "gutter pool".
Given how small is the dataset stored there, we should evaluate both if there are alternative solutions for thumbor's caching needs, as well as the proposed solution (use mcrouter+memcached VMs), is like bringing a steamroller to cover a hole in the ground