Unable to edit/delete Grafana alert
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Krinkle
	Jan 27 2025, 3:04 PM

Description

Error: failed to update rule group: failed to delete rules: database disk image is malformed

E.g. https://grafana-rw.wikimedia.org/alerting/WDAFIjJ4z/edit?returnTo=%2Fd%2F000000402%2Fresourceloader-alerts%3ForgId%3D1%26editPanel%3D7%26tab%3Dalert

Screenshot 2025-01-27 at 15.03.18.png (664×1 px, 197 KB)

I've tried it for a few days and in different browsers. Consistently failing so far.

Related Objects

Mentioned In: T385257: Bump grafana disk space

Event Timeline

Krinkle created this task.Jan 27 2025, 3:04 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 27 2025, 3:04 PM

andrea.denisse claimed this task.Jan 29 2025, 3:16 PM

andrea.denisse subscribed.

lmata edited projects, added SRE Observability (FY2024/2025-Q3); removed SRE Observability.Jan 29 2025, 3:19 PM

fgiunchedi added a project: Observability-Alerting.Jan 30 2025, 2:17 PM

Mentioned in SAL (#wikimedia-operations) [2025-01-30T20:24:00Z] <denisse> Backing up Grafana DB on grafana1002 - T384840

Hi @Krinkle,

This issue occurred due to a malformed GrafanaDB. After restoring the database on both hosts, I was able to delete the rule successfully. Please feel free to reopen this task if needed.

fgiunchedi mentioned this in T385257: Bump grafana disk space.Jan 31 2025, 8:34 AM

	F58291237: Screenshot 2025-01-27 at 15.03.18.png
	Jan 27 2025, 3:04 PM

Unable to edit/delete Grafana alertClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Unable to edit/delete Grafana alert
Closed, ResolvedPublic
Actions