Page MenuHomePhabricator

Enable mcrouter on the memcached servers themselves
Closed, DuplicatePublic

Description

In order for TLS to be set up in a sensible way, there will be mcrouter <=> mcrouter communication. The puppet profile for wancache should changed/used to have mcrouter run all themc* servers (in addition to the app/maintenance servers).

Details

Related Gerrit Patches:

Event Timeline

aaron triaged this task as Normal priority.May 8 2018, 10:48 PM
aaron created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 8 2018, 10:48 PM
Imarlier moved this task from Inbox to Doing on the Performance-Team board.May 14 2018, 7:59 PM

Change 433913 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[operations/puppet@production] [WIP] Enable mcrouter on mediawiki memcached nodes

https://gerrit.wikimedia.org/r/433913

Joe added a subscriber: Joe.May 19 2018, 9:02 AM

Please see https://phabricator.wikimedia.org/T192771 which has a lot of considerations about the mcrouter architecture in production.

I actually decided to set up TLS everywhere and to have mcrouter use some "proxies" in each DC to do the ssl-ssl stuff.

A set of (still partial) patches are here:

https://gerrit.wikimedia.org/r/#/c/431736/

Closing this ticket as a duplicate.

Joe added a comment.May 19 2018, 10:08 AM

To explain my reasoning further: mcrouter needs a non-negligible amount of memory to run as it maintains an internal queue of messages whenever you use something like AllFastRoute or any other route handler that does distribution of keys. This means it can use a significant amount of memory from time to time, and I'd prefer to avoid having any process using a variable amount of memory on the memcached nodes.

We could of course limit the maximum amount of memory mcrouter can use, but that could lead to instabilities, crashes, and lost messages, so I prefer to keep mcrouter in any non-trivial config out of those nodes.

Change 433913 abandoned by Aaron Schulz:
[WIP] Enable mcrouter on mediawiki memcached nodes

Reason:
Superceeded

https://gerrit.wikimedia.org/r/433913

Vvjjkkii renamed this task from Enable mcrouter on the memcached servers themselves to 6bdaaaaaaa.Jul 1 2018, 1:11 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed aaron as the assignee of this task.
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
CommunityTechBot renamed this task from 6bdaaaaaaa to Enable mcrouter on the memcached servers themselves.Jul 2 2018, 6:11 AM
CommunityTechBot closed this task as a duplicate.
CommunityTechBot assigned this task to aaron.
CommunityTechBot lowered the priority of this task from High to Normal.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added subscribers: gerritbot, Aklapper.