Page MenuHomePhabricator

Add prometheus metrics for varnishkafka instances running on caching hosts
Open, NormalPublic

Description

Each varnishkafka is currently set to write stats periodically to a file like /var/cache/varnishkafka/$instance_name.stats.json. We have a root cron that executes every minute logster, that reads the json and pushes the latest metrics to graphite.

Rather than adding HTTP support to varnishkafka to poll the same set of metrics, we could simply write a python script that reads the json file, picks the last datapoints and then exposes the metrics to Prometheus when polled.

Event Timeline

elukey created this task.May 31 2018, 12:21 PM
elukey triaged this task as Normal priority.
Restricted Application added a project: Operations. · View Herald TranscriptMay 31 2018, 12:21 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

As reference, prometheus::node_gdnsd might be an example about how to proceed.

ema moved this task from Triage to Caching on the Traffic board.Jun 4 2018, 9:09 AM
Vvjjkkii renamed this task from Add prometheus metrics for varnishkafka instances running on caching hosts to 1wbaaaaaaa.Jul 1 2018, 1:07 AM
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii removed elukey as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot assigned this task to elukey.
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot renamed this task from 1wbaaaaaaa to Add prometheus metrics for varnishkafka instances running on caching hosts.
CommunityTechBot added a subscriber: Aklapper.
CommunityTechBot triaged this task as Normal priority.Jul 3 2018, 2:24 AM
mforns claimed this task.Oct 19 2018, 1:47 PM
mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.Nov 13 2018, 2:18 PM
elukey added a comment.EditedNov 13 2018, 5:33 PM

@mforns created https://gerrit.wikimedia.org/r/#/admin/projects/operations/debs/prometheus-varnishkafka-exporter

Also added /home/mforns/webrequest.stats.json on stat1007 with an example of librdkafka metrics file that you can use :)

ema added a subscriber: ema.Tue, Mar 26, 2:05 PM
mforns moved this task from In Progress to Paused on the Analytics-Kanban board.Wed, Apr 17, 3:45 PM