When looking at Icinga it's noticable we often have alerts on machines called "cloudvirt*-dev".

These checks often have disabled notifications and when you search in SAL or Phabricator you can't find a reference to ongoing work.

The "dev" part and the disabled notifications make me wonder if it makes sense to have them in Icinga in the first place.

In the past we spent some work on adding parameters to monitoring classes making it possible to disable base checks for certain hosts. For example this was then done via a regex in Hiera for hosts with 'test' in their name.

Wouldn't it make sense to remove these altogether? Hosts that have "dev" in their name seem to be for developing / testing and not production hosts per definition.

Should we really add those checks (to an already overloaded Icinga) only to then regularly disable notifications? It seems a bit of a waste of resources, both on the machine and the humans checking the Icinga web UI.

If we do want to keep them, could we please ACK or downtime these hosts instead of disabling notifications? That way they would not show up as "unhandled". A downtime with a link to a ticket would be ideal.