Page MenuHomePhabricator

Increase swift replication factor for accounts
Open, MediumPublic

Description

The current swift replication factor for accounts is 3. As account was found to be one of the bottlenecks during the recent cache_upload outage, we should increase the replication factor to allow for more fan-out.

Before doing so in eqiad, the change should be tested in codfw, using ssbench to evaluate the impact.

Related incident: https://wikitech.wikimedia.org/wiki/Incident_documentation/2017-01-06_Cache-upload

Event Timeline

ema triaged this task as Medium priority.Jan 24 2017, 2:53 PM
ema added a project: Wikimedia-Incident.

I'm not sure how much this will actually help from the swift side (as opposed to frontend capacity, memcached, etc).
The account dbs are tiny, though, so it seems like a cheap thing to try...