Page MenuHomePhabricator

Make redis/redisdb roles support multiple instances on the same servers
Closed, ResolvedPublic

Description

Dividing up the memory and having multiple redis instances on each server can make an order of magnitude better CPU usage. This could be useful to the job runners for example. It would be nice if the role supported this (including usage with replication) easily.

Event Timeline

aaron raised the priority of this task from to Needs Triage.
aaron updated the task description. (Show Details)
aaron added subscribers: aaron, ori.
chasemp triaged this task as Medium priority.Jun 1 2015, 9:08 PM

@Joe do you have any thoughts here? I only ask as I know you are somewhat experienced with Redis. This seems more or less reasoable to me if 'CPU sharding' is our ceiling.

I'm guessing the client code in question is prepared to support multiple ports? (i.e. this would be 6379, 6374, 6375, etc)

The basics of this, at least on Trusty, are simple. The logistics including ferm, ganglia, graphite, etc become decidedly more complex. While I do think we could have a puppet setup that allows arbitrary numbers of redis instances, that are named resources by port or something, I asked myself why I have not gone down this road before. My previous experiences were that redis is a good virtualization candidate and we scaled redis out horizontally across VM's. I talked with @akosiaris about this a bit wondering if the more idiomatic approach for our environment is this model, and we came away with the conclusion that we need more information on what performance issues we are trying to solve by scaling up redis instances on a single box. In looking at ganglia I am questioning how many redis instances we could actually cohabitate sanely and where the current ceiling is for performance at the moment.

Could someone outline things in terms of specific problems and not the solution here so we can weight the options?

@aaron backatcha so it gets noticed man

RobH mentioned this in Unknown Object (Task).Nov 10 2015, 7:57 PM

Change 253146 had a related patch set uploaded (by Ori.livneh):
Add redis::instance

https://gerrit.wikimedia.org/r/253146

Change 253146 merged by Ori.livneh:
Add redis::instance

https://gerrit.wikimedia.org/r/253146

Change 255175 had a related patch set uploaded (by Ori.livneh):
Migrate rdb1008 to redis::instance

https://gerrit.wikimedia.org/r/255175

Change 255175 merged by Ori.livneh:
Migrate rdb1008 to redis::instance

https://gerrit.wikimedia.org/r/255175