Page MenuHomePhabricator

Move logstash.wikimedia.org (kibana) to an LVS service
Closed, ResolvedPublic

Description

Currently logstash's kibana interface is defined directly in varnish as a pool of randomized backend hostnames. There should be a real service hostname for the internal service like logstash.svc.eqiad.wmnet, which is defined in LVS with pybal controlling the pooling of the 3x backends, and then varnish's configuration should be updated to use that hostname rather than enumerating the backends directly. This involves a handful of complex puppet changes and LVS new-service deploys are always a bit "special" (requiring careful manual restarts).

Details

Related Gerrit Patches:

Event Timeline

BBlack created this task.Apr 12 2016, 3:33 PM
Restricted Application added a project: Operations. · View Herald TranscriptApr 12 2016, 3:33 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ema triaged this task as Medium priority.Apr 12 2016, 6:03 PM

This might be something to look at doing as part of T138328: Upgrade kibana to 4.5.0+.

Gehel added a subscriber: Gehel.Sep 22 2016, 8:07 PM
bd808 renamed this task from Move logstash to an LVS service to Move logstash.wikimedia.org (kibana) to an LVS service.Sep 22 2016, 8:12 PM
bd808 updated the task description. (Show Details)

Change 312342 had a related patch set uploaded (by Gehel):
logstash - DNS entries for LVS service

https://gerrit.wikimedia.org/r/312342

BBlack moved this task from Triage to LoadBalancer on the Traffic board.Sep 30 2016, 2:01 PM

Change 312342 merged by BBlack:
logstash - DNS entries for LVS service

https://gerrit.wikimedia.org/r/312342

Change 315675 had a related patch set uploaded (by Gehel):
kibana - move to an LVS service

https://gerrit.wikimedia.org/r/315675

Change 315676 had a related patch set uploaded (by Gehel):
kibana - activate icinga check on new LVS service

https://gerrit.wikimedia.org/r/315676

Change 315677 had a related patch set uploaded (by Gehel):
kibana - configure varnish to use new LVS service as backend

https://gerrit.wikimedia.org/r/315677

Change 316771 had a related patch set uploaded (by Gehel):
kibana - allow access to both /status and /api/status

https://gerrit.wikimedia.org/r/316771

Change 316772 had a related patch set uploaded (by Gehel):
kibana - change probe URL to /api/status

https://gerrit.wikimedia.org/r/316772

Change 316773 had a related patch set uploaded (by Gehel):
kibana - only allow unauthenticated access to /api/status

https://gerrit.wikimedia.org/r/316773

Change 316774 had a related patch set uploaded (by Gehel):
kibana - move to an LVS service

https://gerrit.wikimedia.org/r/316774

Change 316775 had a related patch set uploaded (by Gehel):
kibana - activate icinga check on new LVS service

https://gerrit.wikimedia.org/r/316775

Change 316776 had a related patch set uploaded (by Gehel):
kibana - configure varnish to use new LVS service as backend

https://gerrit.wikimedia.org/r/316776

Change 315677 abandoned by Gehel:
kibana - configure varnish to use new LVS service as backend

Reason:
Abandonned in favor of https://gerrit.wikimedia.org/r/#/c/316771/

https://gerrit.wikimedia.org/r/315677

Change 315676 abandoned by Gehel:
kibana - activate icinga check on new LVS service

Reason:
Abandonned in favor of https://gerrit.wikimedia.org/r/#/c/316771/

https://gerrit.wikimedia.org/r/315676

Change 315675 abandoned by Gehel:
kibana - move to an LVS service

Reason:
Abandonned in favor of https://gerrit.wikimedia.org/r/#/c/316771/

https://gerrit.wikimedia.org/r/315675

Change 316771 merged by Gehel:
kibana - allow access to both /status and /api/status

https://gerrit.wikimedia.org/r/316771

Change 316772 merged by Gehel:
kibana - change probe URL to /api/status

https://gerrit.wikimedia.org/r/316772

Change 316773 merged by Gehel:
kibana - only allow unauthenticated access to /api/status

https://gerrit.wikimedia.org/r/316773

Mentioned in SAL (#wikimedia-operations) [2016-10-20T16:26:57Z] <gehel> deploying new LVS service for kibana - T132458

Change 316774 merged by Gehel:
kibana - move to an LVS service

https://gerrit.wikimedia.org/r/316774

Change 316775 merged by Gehel:
kibana - activate icinga check on new LVS service

https://gerrit.wikimedia.org/r/316775

Change 316776 merged by Gehel:
kibana - configure varnish to use new LVS service as backend

https://gerrit.wikimedia.org/r/316776

BBlack closed this task as Resolved.Oct 21 2016, 10:31 AM
BBlack assigned this task to Gehel.