icinga UNKNOWN Varnishkafka Delivery Errors / varnishkafka data not in graphite
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Dzahn
	Mar 17 2015, 3:07 PM

Description

we have a lot of UNKNOWNs in Icinga from:

Varnishkafka Delivery Errors per minute

they are either "UNKNOWN: No valid datapoints found" or "UNKNOWN: More than half of the datapoints are undefined"

I reopened T76342 for this, but:

< ottomata> mutante: i think that is a new issue, that one was for check_ganglia. this one is from graphite, and the data is really not in graphite

	Subject	Repo	Branch	Lines +/-
	graphite: add error alerts	operations/puppet	production	+50 -0
	graphite: increase relay queue size	operations/puppet	production	+1 -0

Mentioned In: rOPUP2625aac65b1c: graphite: add error alerts
rOPUP22ed5ac71568: graphite: increase relay queue size
T92967: hhvm - Icinga - UNKNOWN queue size / busy threads
T76342: Fix Varnishkafka delivery error icinga warning
Mentioned Here: T92967: hhvm - Icinga - UNKNOWN queue size / busy threads
T76342: Fix Varnishkafka delivery error icinga warning

Dzahn raised the priority of this task from to Needs Triage.

Dzahn updated the task description. (Show Details)

Dzahn subscribed.

same issue on T92967 but for HHVM things (graphite issue?)

MC8 subscribed.

confirmed it is a graphite issue, namely the carbon-relay queue is too small now, will followup with a fix

Change 197344 had a related patch set uploaded (by Filippo Giunchedi):
graphite: increase relay queue size

Change 197344 merged by Filippo Giunchedi:
graphite: increase relay queue size

fix merged, seems to have recovered, pending creation of related alarms

Change 197352 had a related patch set uploaded (by Filippo Giunchedi):
graphite: add error alerts

Change 197352 merged by Filippo Giunchedi:
graphite: add error alerts

alarms created, resolving