The simplest way to solve our problems with elasticsearch and the single type restriction would be to move namespace to the meta store, and add a new index for archive. Archives are small, at most about 50k documents per wiki, so only needs a single shard. It still would need to exist for 900 wikis with a primary and 2 replicas. If we want to reject this solution because it's not going to work, that should be easy enough to prove.
- Devise a test that measures the latency of various master operations.
- Develop test on a single node local elasticsearch cluster
- If promising, run test on hot-spare production cluster
- If still promising, run test on live production cluster to see full-load impact
Cluster operations to measure:
- Index Create/Delete
- Index settings/mapping update
- Move shard between nodes (if >1 node)
- cluster settings update
- Time to read outputs like cluster state, _cat/shards, _cat/indices
- probably more