Comprehensive monitoring / alerting for labstore* instances
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	yuvipanda
	Mar 31 2015, 7:24 PM

Description

Should have alerts for any issues with labstore*:

	Subject	Repo	Branch	Lines +/-
	labs: Alert on high load in labstore*	operations/puppet	production	+11 -0
	labs: Add monitoring for high iowait on labstore instances	operations/puppet	production	+8 -0

Mentioned In: rOPUP9b19153a78cf: labs: Alert on high load in labstore*
rOPUPef6b09299bf3: labs: Add monitoring for high iowait on labstore instances

yuvipanda raised the priority of this task from to Needs Triage.

yuvipanda updated the task description. (Show Details)

Change 201591 had a related patch set uploaded (by Yuvipanda):
labs: Add monitoring for high iowait on labstore instances

We already have checks for network saturation as well.

Change 201591 merged by Yuvipanda:
labs: Add monitoring for high iowait on labstore instances

Change 201618 had a related patch set uploaded (by Yuvipanda):
labs: Alert on high load in labstore*

There's already network saturation alerts.

We should probably make these paging as well, though.

Also, I'm wondering if there should be *all* graphite alerts, or we should have active alerts as well. hmm.

Change 201618 merged by Yuvipanda:
labs: Alert on high load in labstore*

Alright, so I'm going to consider this 'done' for now. More checks as warrented.