The task tracks setting up M.O.S.S. or misc object storage service. The service is meant to host all existing (and new) object storage use cases that have accumulated over the years in the media storage cluster (listed at {T264291} with more details)
The cluster will have at least 3x replication, will be spanning across eqiad and codfw, and offer `S3` + `swift` API compatibility (i.e. same as the thanos cluster).
[] Evaluate whether we need encrypted backend traffic across datacenters for the cluster (likely ipsec)
[] Decide on initial storage policies (replication factor, ssd/hdd, site-local vs global, which should be default, etc)
[] Bring frontends online: T275513 T275511
[] Bring backends online: T276642 T276637
[] Bring up service IPs / LVS and certs
[] Bring up dashboards/monitoring/alerting
Once the service/cluster is up we can start migrating users / use cases (in a different task, TBD)