Page MenuHomePhabricator
Create Task
Maniphest T132457

Move wdqs to an LVS service
Closed, ResolvedPublic
Actions

Description

Currently wdqs is defined directly in varnish as a pool of randomized backend hostnames. There should be a real service hostname for the internal service like wdqs.svc.eqiad.wmnet, which is defined in LVS with pybal controlling the pooling of the 3x backends, and then varnish's configuration should be updated to use that hostname rather than enumerating the backends directly. This involves a handful of complex puppet changes and LVS new-service deploys are always a bit "special" (requiring careful manual restarts).

Details

SubjectRepoBranchLines +/-
wdqs - configure varnish to use LVS service as backendoperations/puppetproduction+1 -12
wdqs - add icinga check for LVS servicesoperations/puppetproduction+7 -0
wdqs - LVS configurationoperations/puppetproduction+46 -0
wdqs LVS DNS entriesoperations/dnsmaster+6 -0
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedBBlackT147844 Standardize varnish applayer backend definitions
 ResolvedBBlackT132457 Move wdqs to an LVS service

Event Timeline

BBlack created this task.Apr 12 2016, 3:32 PM
Restricted Application added projects: SRE, Wikidata, Discovery-ARCHIVED. · View Herald TranscriptApr 12 2016, 3:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• ema triaged this task as Medium priority.Apr 12 2016, 6:04 PM
Smalyshev moved this task from Incoming to Watching / Waiting on the Wikidata-Query-Service board.Apr 18 2016, 11:18 PM
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.Apr 21 2016, 1:47 PM
Gehel mentioned this in T146158: Configure varnish to include wdqs nodes in codfw.Sep 20 2016, 4:00 PM
Gehel added a project: Discovery-Wikidata-Query-Service-Sprint.Sep 22 2016, 9:49 AM
Gehel moved this task from Backlog to In progress on the Discovery-Wikidata-Query-Service-Sprint board.
gerritbot subscribed.Sep 22 2016, 9:50 AM

Change 312216 had a related patch set uploaded (by Gehel):
wdqs LVS DNS entries

https://gerrit.wikimedia.org/r/312216

gerritbot added a project: Patch-For-Review.Sep 22 2016, 9:50 AM
gerritbot added a comment.Sep 22 2016, 10:30 AM

Change 312223 had a related patch set uploaded (by Gehel):
wdqs - LVS configuration

https://gerrit.wikimedia.org/r/312223

gerritbot added a comment.Sep 22 2016, 10:30 AM

Change 312224 had a related patch set uploaded (by Gehel):
wdqs - add icinga check for LVS services

https://gerrit.wikimedia.org/r/312224

gerritbot added a comment.Sep 22 2016, 10:30 AM

Change 312225 had a related patch set uploaded (by Gehel):
wdqs - configure varnish to use LVS service as backend

https://gerrit.wikimedia.org/r/312225

BBlack moved this task from Backlog to LoadBalancer on the Traffic board.Sep 30 2016, 2:01 PM
gerritbot added a comment.Oct 5 2016, 1:36 PM

Change 312216 merged by Gehel:
wdqs LVS DNS entries

https://gerrit.wikimedia.org/r/312216

Stashbot subscribed.Oct 5 2016, 1:46 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-05T13:46:18Z] <gehel> deploying new LVS configuration for WDQS service - T132457

gerritbot added a comment.Oct 5 2016, 2:08 PM

Change 312223 merged by Gehel:
wdqs - LVS configuration

https://gerrit.wikimedia.org/r/312223

Stashbot added a comment.Oct 5 2016, 2:11 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-05T14:11:49Z] <gehel> restarting pybal on lvs1006 - T132457

Stashbot added a comment.Oct 5 2016, 2:15 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-05T14:15:00Z] <gehel> restarting pybal on lvs1009 - T132457

Stashbot added a comment.Oct 5 2016, 2:17 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-05T14:17:31Z] <gehel> restarting pybal on lvs1012 - T132457

Stashbot added a comment.Oct 5 2016, 2:18 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-05T14:18:53Z] <gehel> restarting pybal on lvs1003 - T132457

Stashbot added a comment.Oct 5 2016, 2:22 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-05T14:22:27Z] <gehel> restarting pybal on lvs1006 - T132457

Stashbot added a comment.Oct 5 2016, 2:27 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-05T14:27:40Z] <gehel> restarting pybal on lvs1003 - T132457

gerritbot added a comment.Oct 5 2016, 2:36 PM

Change 312224 merged by Gehel:
wdqs - add icinga check for LVS services

https://gerrit.wikimedia.org/r/312224

BBlack added a parent task: T147844: Standardize varnish applayer backend definitions.Oct 11 2016, 12:35 PM
Smalyshev moved this task from Watching / Waiting to Operations/SRE on the Wikidata-Query-Service board.Oct 11 2016, 5:53 PM
gerritbot added a comment.Oct 12 2016, 5:33 PM

Change 312225 merged by BBlack:
wdqs - configure varnish to use LVS service as backend

https://gerrit.wikimedia.org/r/312225

gerritbot added a comment.Oct 12 2016, 5:37 PM

Change 315528 had a related patch set uploaded (by BBlack):
cache_misc: removed wdqs probe ref T132457

https://gerrit.wikimedia.org/r/315528

gerritbot added a comment.Oct 12 2016, 5:37 PM

Change 315528 merged by BBlack:
cache_misc: removed wdqs probe ref T132457

https://gerrit.wikimedia.org/r/315528

BBlack closed this task as Resolved.Oct 12 2016, 5:39 PM
BBlack claimed this task.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits