Page MenuHomePhabricator

Create globally-unique varnish cache cluster port/instancename mappings
Closed, DeclinedPublic

Description

All 2layer varnish instances are currently configured as follows:

frontend instance:
  name: "frontend"
  port: 80
  admin_port: 6082
backend instance:
  name: "" (default)
  port: 3128
  admin_port: 6083

For several related reasons, we'd like these to be unique per-cluster-type and per-instance-type. The listening port is particularly problematic in that it will require several interrelated changes over time, so it will be set to listen on both old and new ports for a transition period that could last a while.

The new instance names will all be non-default for the backends, and include the cluster name as a prefix, as in "text-frontend" and "text-backend".

new admin_port (fe, be):

text:    6082, 6083
upload:  6084, 6085
misc:    6086, 6087
maps:    6088, 6089

new port (fe, be - old 80/3128 still configured as well):

text:    5082, 5083
upload:  5084, 5085
misc:    5086, 5087
maps:    5088, 5089

Event Timeline

BBlack raised the priority of this task from to Needs Triage.
BBlack updated the task description. (Show Details)
BBlack added a project: Traffic.
BBlack added a subscriber: BBlack.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald Transcript

Further notes:

The changes to the instance name and admin_port should be machine-local in effect - they're not going to cause actual traffic havoc, but they do cause issues with interrelated local software like traffic loggers and the reload-vcl script. Therefore, it would be best to schedule those changes to coincide with a depooled reboot, or at least a depool window during which all varnish-related daemons can be restarted together.

The addition of the new listening port should have no real effect, we just need to wait for it to be completed (varnish restarts) across clusters before moving on with other blocked tasks.

Change 255107 had a related patch set uploaded (by BBlack):
varnish: refactor instance parameters (no-op)

https://gerrit.wikimedia.org/r/255107

Change 255107 merged by BBlack:
varnish: refactor instance parameters (no-op)

https://gerrit.wikimedia.org/r/255107

BBlack triaged this task as Medium priority.

Change 256260 had a related patch set uploaded (by BBlack):
cache_misc: switch to new port/instance mapping

https://gerrit.wikimedia.org/r/256260

Change 256259 had a related patch set uploaded (by BBlack):
varnish: ability to remove a named instance

https://gerrit.wikimedia.org/r/256259

Change 256279 had a related patch set uploaded (by BBlack):
update varnish directors for full instance names

https://gerrit.wikimedia.org/r/256279

Change 256280 had a related patch set uploaded (by BBlack):
cache_maps: switch to new port/instance mapping

https://gerrit.wikimedia.org/r/256280

Change 256259 merged by BBlack:
varnish: ability to remove a named instance

https://gerrit.wikimedia.org/r/256259

Change 256279 merged by BBlack:
update varnish directors for full instance names

https://gerrit.wikimedia.org/r/256279

Change 256260 abandoned by BBlack:
cache_misc: switch to new port/instance mapping

Reason:
This is stale, can re-do later if we ever get back around to this...

https://gerrit.wikimedia.org/r/256260

Change 256280 abandoned by BBlack:
cache_maps: switch to new port/instance mapping

Reason:
This is stale, can re-do later if we ever get back around to this...

https://gerrit.wikimedia.org/r/256280

BBlack updated the task description. (Show Details)
BBlack set Security to None.
BBlack updated the task description. (Show Details)

We did some interesting refactoring here that helped with some other related issues, but the main goal here is no longer on our radar (instance naming and ports are fine as they are; we won't be merging cache daemons onto shared hardware; and ATS Is Coming).