Page MenuHomePhabricator
Create Task
Maniphest T132458

Move logstash.wikimedia.org (kibana) to an LVS service
Closed, ResolvedPublic
Actions

Description

Currently logstash's kibana interface is defined directly in varnish as a pool of randomized backend hostnames. There should be a real service hostname for the internal service like logstash.svc.eqiad.wmnet, which is defined in LVS with pybal controlling the pooling of the 3x backends, and then varnish's configuration should be updated to use that hostname rather than enumerating the backends directly. This involves a handful of complex puppet changes and LVS new-service deploys are always a bit "special" (requiring careful manual restarts).

Details

SubjectRepoBranchLines +/-
kibana - configure varnish to use new LVS service as backendoperations/puppetproduction+2 -14
kibana - activate icinga check on new LVS serviceoperations/puppetproduction+5 -0
kibana - move to an LVS serviceoperations/puppetproduction+38 -0
kibana - only allow unauthenticated access to /api/statusoperations/puppetproduction+1 -4
kibana - change probe URL to /api/statusoperations/puppetproduction+1 -1
kibana - allow access to both /status and /api/statusoperations/puppetproduction+4 -1
kibana - move to an LVS serviceoperations/puppetproduction+38 -0
kibana - activate icinga check on new LVS serviceoperations/puppetproduction+7 -2
kibana - configure varnish to use new LVS service as backendoperations/puppetproduction+2 -14
logstash - DNS entries for LVS serviceoperations/dnsmaster+4 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

BBlack created this task.Apr 12 2016, 3:33 PM
Restricted Application added a project: SRE. · View Herald TranscriptApr 12 2016, 3:33 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• ema triaged this task as Medium priority.Apr 12 2016, 6:03 PM
bd808 added a comment.Jun 21 2016, 5:30 PM

This might be something to look at doing as part of T138328: Upgrade kibana to 4.5.0+.

bd808 added a comment.Jun 21 2016, 5:36 PM

Related: T113104: Set up a service IP for logstash

Gehel subscribed.Sep 22 2016, 8:07 PM
bd808 renamed this task from Move logstash to an LVS service to Move logstash.wikimedia.org (kibana) to an LVS service.Sep 22 2016, 8:12 PM
bd808 updated the task description. (Show Details)
gerritbot subscribed.Sep 22 2016, 9:10 PM

Change 312342 had a related patch set uploaded (by Gehel):
logstash - DNS entries for LVS service

https://gerrit.wikimedia.org/r/312342

gerritbot added a project: Patch-For-Review.Sep 22 2016, 9:10 PM
BBlack moved this task from Backlog to LoadBalancer on the Traffic board.Sep 30 2016, 2:01 PM
BBlack added a parent task: T147844: Standardize varnish applayer backend definitions.Oct 11 2016, 12:35 PM
gerritbot added a comment.Oct 12 2016, 6:10 PM

Change 312342 merged by BBlack:
logstash - DNS entries for LVS service

https://gerrit.wikimedia.org/r/312342

gerritbot added a comment.Oct 13 2016, 2:08 PM

Change 315675 had a related patch set uploaded (by Gehel):
kibana - move to an LVS service

https://gerrit.wikimedia.org/r/315675

gerritbot added a comment.Oct 13 2016, 2:08 PM

Change 315676 had a related patch set uploaded (by Gehel):
kibana - activate icinga check on new LVS service

https://gerrit.wikimedia.org/r/315676

gerritbot added a comment.Oct 13 2016, 2:08 PM

Change 315677 had a related patch set uploaded (by Gehel):
kibana - configure varnish to use new LVS service as backend

https://gerrit.wikimedia.org/r/315677

gerritbot added a comment.Oct 19 2016, 11:55 AM

Change 316771 had a related patch set uploaded (by Gehel):
kibana - allow access to both /status and /api/status

https://gerrit.wikimedia.org/r/316771

gerritbot added a comment.Oct 19 2016, 11:55 AM

Change 316772 had a related patch set uploaded (by Gehel):
kibana - change probe URL to /api/status

https://gerrit.wikimedia.org/r/316772

gerritbot added a comment.Oct 19 2016, 11:55 AM

Change 316773 had a related patch set uploaded (by Gehel):
kibana - only allow unauthenticated access to /api/status

https://gerrit.wikimedia.org/r/316773

gerritbot added a comment.Oct 19 2016, 11:55 AM

Change 316774 had a related patch set uploaded (by Gehel):
kibana - move to an LVS service

https://gerrit.wikimedia.org/r/316774

gerritbot added a comment.Oct 19 2016, 11:55 AM

Change 316775 had a related patch set uploaded (by Gehel):
kibana - activate icinga check on new LVS service

https://gerrit.wikimedia.org/r/316775

gerritbot added a comment.Oct 19 2016, 11:55 AM

Change 316776 had a related patch set uploaded (by Gehel):
kibana - configure varnish to use new LVS service as backend

https://gerrit.wikimedia.org/r/316776

gerritbot added a comment.Oct 19 2016, 11:56 AM

Change 315677 abandoned by Gehel:
kibana - configure varnish to use new LVS service as backend

Reason:
Abandonned in favor of https://gerrit.wikimedia.org/r/#/c/316771/

https://gerrit.wikimedia.org/r/315677

gerritbot added a comment.Oct 19 2016, 11:56 AM

Change 315676 abandoned by Gehel:
kibana - activate icinga check on new LVS service

Reason:
Abandonned in favor of https://gerrit.wikimedia.org/r/#/c/316771/

https://gerrit.wikimedia.org/r/315676

gerritbot added a comment.Oct 19 2016, 11:56 AM

Change 315675 abandoned by Gehel:
kibana - move to an LVS service

Reason:
Abandonned in favor of https://gerrit.wikimedia.org/r/#/c/316771/

https://gerrit.wikimedia.org/r/315675

gerritbot added a comment.Oct 20 2016, 1:44 PM

Change 316771 merged by Gehel:
kibana - allow access to both /status and /api/status

https://gerrit.wikimedia.org/r/316771

gerritbot added a comment.Oct 20 2016, 1:50 PM

Change 316772 merged by Gehel:
kibana - change probe URL to /api/status

https://gerrit.wikimedia.org/r/316772

gerritbot added a comment.Oct 20 2016, 2:58 PM

Change 316773 merged by Gehel:
kibana - only allow unauthenticated access to /api/status

https://gerrit.wikimedia.org/r/316773

Stashbot subscribed.Oct 20 2016, 4:26 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-20T16:26:57Z] <gehel> deploying new LVS service for kibana - T132458

gerritbot added a comment.Oct 20 2016, 4:27 PM

Change 316774 merged by Gehel:
kibana - move to an LVS service

https://gerrit.wikimedia.org/r/316774

gerritbot added a comment.Oct 20 2016, 5:09 PM

Change 316775 merged by Gehel:
kibana - activate icinga check on new LVS service

https://gerrit.wikimedia.org/r/316775

gerritbot added a comment.Oct 20 2016, 6:50 PM

Change 316776 merged by Gehel:
kibana - configure varnish to use new LVS service as backend

https://gerrit.wikimedia.org/r/316776

BBlack closed this task as Resolved.Oct 21 2016, 10:31 AM
BBlack assigned this task to Gehel.
fgiunchedi merged a task: T113104: Set up a service IP for logstash.Jul 3 2018, 4:28 PM
fgiunchedi added subscribers: • GWicke, Matanya, • mobrovac and 2 others.
fgiunchedi added a project: observability.Aug 19 2019, 2:32 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits