Right now https://quarry.wmcloud.org/ is displaying an error page. There is nothing on alertmanager indicating that anything is wrong with Quarry.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | BUG REPORT | Andrew | T392107 quarry.wmcloud.org: "This web service cannot be reached" due to redis pod running out of disk space | ||
| Resolved | BUG REPORT | taavi | T392138 No alerting for quarry |
Event Timeline
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2025-04-17T10:55:33Z] <taavi> deploying T392138 T392141 T392143 patches
Comment Actions
MariaDB [prometheusconfig]> insert into alerts (project_id, name, expr, duration, severity, annotations) values (16, 'Down', 'up{project="quarry",job="app"} == 0', '5m', 'critical', '{"summary": "Quarry application is unreachable"}');
Query OK, 1 row affected (0.033 sec)