We did some work to get openapi / swagger metrics into Prometheus as part of T205870. With Alertmanager and alerts.git being a reality nowadays we should clean up the Icinga checks and move to Prometheus-based alerts.
While at it I think it'd be beneficial to at least brainstorm on the situation of modules/lvs/manifests/monitor_services.pp which is basically a separate list of openapi-based services, and IMHO it should be part of service::catalog instead.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | colewhite | T320620 Port openapi/swagger checks/alerts to Prometheus | |||
| Resolved | colewhite | T346893 Investigate swagger-exporter failures |
Change 916914 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] prometheus: generate swagger targets from service catalog
Change 918547 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/alerts@master] team-sre: add openapi/swagger alerts
Change 919331 had a related patch set uploaded (by Filippo Giunchedi; author: Filippo Giunchedi):
[operations/puppet@production] prometheus: don't fail on unknown blackbox probe type
Change 919331 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: don't fail on unknown blackbox probe type
Change 923576 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] prometheus: don't add empty targets
Change 923576 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: don't add empty targets
Change 916914 merged by Cwhite:
[operations/puppet@production] prometheus: generate swagger targets from service catalog
Change 924144 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] prometheus: remove invalid cluster key
Change 924144 merged by Cwhite:
[operations/puppet@production] prometheus: remove invalid cluster key
Change 924145 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] prometheus: disable new swagger job
Change 924145 merged by Cwhite:
[operations/puppet@production] prometheus: disable new swagger job
Change 925106 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] prometheus: ensure absent invalid swagger targets file
Change 925106 merged by Cwhite:
[operations/puppet@production] prometheus: ensure absent invalid swagger targets file
Change 925113 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] opensearch_dashboards: fix package name typo
Change 925113 merged by Cwhite:
[operations/puppet@production] opensearch_dashboards: fix package name typo
Change 925117 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] prometheus: re-enable swagger jobs from service catalog
Change 925117 merged by Cwhite:
[operations/puppet@production] prometheus: re-enable swagger jobs from service catalog
Change 925118 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] prometheus: fix swagger job relabel configs
Change 925118 merged by Cwhite:
[operations/puppet@production] prometheus: fix swagger job relabel configs
Change 925119 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] prometheus: add external swagger checks to all sites
Change 925120 had a related patch set uploaded (by Cwhite; author: Cwhite):
[operations/puppet@production] lvs: remove lvs::monitor_services
Change 925119 merged by Cwhite:
[operations/puppet@production] prometheus: add external swagger checks to all sites
Change 918547 merged by jenkins-bot:
[operations/alerts@master] team-sre: add openapi/swagger alerts
Change 925120 merged by Cwhite:
[operations/puppet@production] lvs: remove lvs::monitor_services