Page MenuHomePhabricator

various graphite based monitoring checks broken (memcached, parsoid, restbase, eventlogging..)
Closed, DuplicatePublic

Description

all of the following icinga checks that check graphite data are broken with undefined datapoints:

https://icinga.wikimedia.org/cgi-bin/icinga/status.cgi?host=graphite1001&nostatusheader

these are all reported as UNKNOWN in Icinga and have been broken like this for a long time.

we should remove them or fix them but stop just ignoring them


MediaWiki memcached error rate

UNKNOWN 2015-12-23 19:47:37 0d 0h 0m 32s 1/3 UNKNOWN: More than half of the datapoints are undefined

Parsoid HTTP 5xx reqs/min

UNKNOWN 2015-12-23 19:46:40 0d 0h 3m 25s 2/3 UNKNOWN: More than half of the datapoints are undefined

RESTBase Cassandra highest dropped message rate http://grafana.wikimedia.org/#/dashboard/db/restbase-cassandra-dropped-messages

UNKNOWN 2015-12-23 19:46:39 0d 0h 3m 28s 2/3 UNKNOWN: More than half of the datapoints are undefined

RESTBase Cassandra highest total hints http://grafana.wikimedia.org/#/dashboard/db/restbase-cassandra-storage

UNKNOWN 2015-12-23 19:47:17 0d 0h 0m 52s 1/3 UNKNOWN: More than half of the datapoints are undefined

Throughput of event logging NavigationTiming events

UNKNOWN 2015-12-23 19:46:30 29d 9h 25m 36s 3/3 UNKNOWN: No valid datapoints found

Uploads HTTP 5xx reqs/min

UNKNOWN 2015-12-23 19:46:21 27d 3h 43m 58s 3/3 UNKNOWN: execution of the check script exited with exception list index out of range