Dividing up the memory and having multiple redis instances on each server can make an order of magnitude better CPU usage. This could be useful to the job runners for example. It would be nice if the role supported this (including usage with replication) easily.
Description
Details
Subject | Repo | Branch | Lines +/- | |
---|---|---|---|---|
Migrate rdb1008 to redis::instance | operations/puppet | production | +29 -10 | |
Add redis::instance | operations/puppet | production | +114 -2 |
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | ori | T100714 Make redis/redisdb roles support multiple instances on the same servers | |||
Resolved | yuvipanda | T118704 Upstart support for redis::instance | |||
Resolved | hashar | T121396 puppet keep trying to restart redis because upstart track wrong PID |
Event Timeline
@Joe do you have any thoughts here? I only ask as I know you are somewhat experienced with Redis. This seems more or less reasoable to me if 'CPU sharding' is our ceiling.
I'm guessing the client code in question is prepared to support multiple ports? (i.e. this would be 6379, 6374, 6375, etc)
The basics of this, at least on Trusty, are simple. The logistics including ferm, ganglia, graphite, etc become decidedly more complex. While I do think we could have a puppet setup that allows arbitrary numbers of redis instances, that are named resources by port or something, I asked myself why I have not gone down this road before. My previous experiences were that redis is a good virtualization candidate and we scaled redis out horizontally across VM's. I talked with @akosiaris about this a bit wondering if the more idiomatic approach for our environment is this model, and we came away with the conclusion that we need more information on what performance issues we are trying to solve by scaling up redis instances on a single box. In looking at ganglia I am questioning how many redis instances we could actually cohabitate sanely and where the current ceiling is for performance at the moment.
Could someone outline things in terms of specific problems and not the solution here so we can weight the options?
Change 255175 had a related patch set uploaded (by Ori.livneh):
Migrate rdb1008 to redis::instance