Page MenuHomePhabricator

Declarative configuration for varnish services and backends
Closed, DuplicatePublic

Description

Related to: T110472 (parsoidcache eventual decom) and T93428 (streamline service deploy)

For services, I'd like to move away from having to define custom VCL code stanzas and puppet backend-related fragments, and get to a place where we can define the service's varnish-layer needs in an abstract data structure recording some standard options for how the service behaves and its basic attributes, from which standardized VCL fragments are templated.

Hypothetical example (there are lots of holes in this if you stare at it long enough, it's just meant to approximate the direction of things):

text_services => {
  cxserver => {
    recv => {
      host_eq => 'cxserver.wikimedia.org',
      # or path_re => '/api/cx_v1/....'
    },
    cache_policy => 'pass' # no caching at all
    # cache_policy => 'default' # normal, obeys CC-headers
    # cache_policy => { ... future configurable declarative complexities ... }
    backend => {
      method => 'chash' # or round-robin, etc...
      hosts => [ 'cxserver.svc.eqiad.wmnet' ],
      port => 8000,
      probe_options => {
        timeout => 30s,
        max_connections => 1000,
      },
    },
  },
}

The abstraction here is meant to treat the entire varnish cache layer of our infrastructure (ignoring internal details like layers and tiers) as a black box, and simply define the rules for how the cache layer as a whole handles requests for a certain distinct service. We can start out only implementing a limited set of options for the services that are simplest in terms of Varnish config needs, and migrate more services into this scheme (and away from current manual VCL code + puppet frags) as this mechanism gains the ability to handle more-complex cases. As things stand today, most of the services being handled by the parsoidcache (other than parsoid itself) currently, as well as most of the services flowing through the misc cluster, could be handled by a very basic version of the above without a lot of complexity-handling.

Perhaps ideally, this can be factored to remove some of the duplication between the backend data above and the inbound-side LVS definitions for the same internal service to really get the configuration minimized, but starting with just the abstraction of the Varnish bits is a good initial step in that direction (and not all backends are actually LVS-based anyways).

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
BBlack raised the priority of this task from to Medium.Aug 28 2015, 6:55 PM
BBlack updated the task description. (Show Details)
BBlack added projects: Traffic, Services.
BBlack added subscribers: BBlack, faidon, mobrovac and 2 others.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
BBlack renamed this task from Define a standardized config mechanism for exposing services through the varnish to Define a standardized config mechanism for exposing services through varnish.Aug 28 2015, 6:56 PM
BBlack set Security to None.

/me likes this direction. From the looks of it, ideally we should be able to integrate something like that *at least* in the service::node ops/puppet module, or in @akosiaris' new_wmf_service.py helper script.

(sorry, no ref links as I'm sending this from my phone)

Change 294495 had a related patch set uploaded (by BBlack):
[WIP] - move app_directors logic to puppet parser func

https://gerrit.wikimedia.org/r/294495

BBlack renamed this task from Define a standardized config mechanism for exposing services through varnish to Declarative configuration for varnish services and backends.Jul 22 2016, 2:32 PM

(re-titled because I could never find the old title when I looked for it!)

Change 300560 had a related patch set uploaded (by BBlack):
text VCL: refactor backend selection

https://gerrit.wikimedia.org/r/300560

Change 300561 had a related patch set uploaded (by BBlack):
Text VCL: split X-Wikimedia-Debug from the rest

https://gerrit.wikimedia.org/r/300561

Change 294495 abandoned by BBlack:
[WIP] - move app_directors logic to puppet parser func

Reason:
Different approach for now, may revisit this concept later if necc

https://gerrit.wikimedia.org/r/294495

Change 300574 had a related patch set uploaded (by BBlack):
VCL backends work 1/N [WIP]

https://gerrit.wikimedia.org/r/300574

Change 300579 had a related patch set uploaded (by BBlack):
VCL backends 2/N: sort misc req_handling

https://gerrit.wikimedia.org/r/300579

Change 300580 had a related patch set uploaded (by BBlack):
VCL backends 3/N: no need for (?i) on planet

https://gerrit.wikimedia.org/r/300580

Change 300581 had a related patch set uploaded (by BBlack):
VCL backends 4/N: add force-pass support

https://gerrit.wikimedia.org/r/300581

Change 301650 had a related patch set uploaded (by BBlack):
cache_misc: no need for (?i) on planet regex

https://gerrit.wikimedia.org/r/301650

Change 301650 merged by BBlack:
cache_misc: no need for (?i) on planet regex

https://gerrit.wikimedia.org/r/301650

Change 300580 abandoned by BBlack:
VCL backends 3/N: no need for (?i) on planet

Reason:
refactored out

https://gerrit.wikimedia.org/r/300580

Change 300574 abandoned by BBlack:
VCL backends 1/N [WIP]

https://gerrit.wikimedia.org/r/300574

Change 300579 abandoned by BBlack:
VCL backends 2/N: sort misc req_handling

https://gerrit.wikimedia.org/r/300579

Change 300581 abandoned by BBlack:
VCL backends 3/N: add force-pass support

https://gerrit.wikimedia.org/r/300581

Change 300574 restored by BBlack:
VCL backends 1/N [WIP]

Reason:
reworking on top of some new things, a little simpler!

https://gerrit.wikimedia.org/r/300574

Change 300579 restored by BBlack:
VCL backends 2/N: sort misc req_handling

Reason:
reworking on top of some new things, a little simpler!

https://gerrit.wikimedia.org/r/300579

Change 324942 had a related patch set uploaded (by BBlack):
VCL refactor: split cache/app backend support

https://gerrit.wikimedia.org/r/324942

Change 300581 restored by BBlack:
VCL backends 3/N: add force-pass support

Reason:
Redoing these a little differently

https://gerrit.wikimedia.org/r/300581

Change 325798 had a related patch set uploaded (by BBlack):
Varnish: remove "varnish-be-rand" conftool service

https://gerrit.wikimedia.org/r/325798

Change 300655 had a related patch set uploaded (by BBlack):
cache_misc req_handling: subpaths and defaulting

https://gerrit.wikimedia.org/r/300655

Change 324942 merged by BBlack:
VCL refactor: split cache/app backend support

https://gerrit.wikimedia.org/r/324942

Change 326471 had a related patch set uploaded (by BBlack):
Varnish: remove "varnish-be-rand" conftool service 2/2

https://gerrit.wikimedia.org/r/326471

Change 325798 merged by BBlack:
Varnish: remove "varnish-be-rand" conftool service 1/2

https://gerrit.wikimedia.org/r/325798

Change 326471 merged by BBlack:
Varnish: remove "varnish-be-rand" conftool service 2/2

https://gerrit.wikimedia.org/r/326471

Change 300574 merged by BBlack:
cache_misc app_directors/req_handling split

https://gerrit.wikimedia.org/r/300574

Change 300579 merged by BBlack:
cache_misc req_handling: sort entries

https://gerrit.wikimedia.org/r/300579

Change 300655 abandoned by BBlack:
cache_misc req_handling: subpaths and defaulting

Reason:
replaced

https://gerrit.wikimedia.org/r/300655

Change 300581 merged by BBlack:
cache_misc req_handling: subpaths, cache policy, defaulting

https://gerrit.wikimedia.org/r/300581