We've been discussing splitting traffic across datacenters in order to have more capacity for ORES.
This task is done when we've identified the strategy we'd like to follow and created the tasks in order to implement it.
Lots of relevant discussion has already taken place at T139372: Set up oresrdb redis node in codfw
So I thought a bit about it and come up with the following alternative solution
- the celery side of the redis for ores MUST NOT be replicated between datacenters
- We can set up nutcracker to act as a proxy between ores and the cache redis instances in both datacenters, so that all ores nodes share all the caches, and
- set up ipsec between where nutcracker is and the oresrdbXXXX that actually stores the data.
but honestly for now duplicating the precaching jobs to both DCs is ok.