Maniphest T400981

Check alerting for Loki
Open, MediumPublic
Actions

Assigned To

None

Authored By

	taavi
	Aug 1 2025, 9:13 AM

Tags

Referenced Files

None

Subscribers

Description

Add Prometheus alerts to check Loki and Alloy health basics
Ensure loading logs from Loki is covered by Toolforge automated tests

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• Bstorm	T126083 overhaul labstore setup [tracking]
		Resolved		• GTirloni	T216988 labstore1004 - DISK CRITICAL - free space: /srv/tools 115904 MB (1% inode=79%):
		Resolved		• Bstorm	T217993 2019-03-10: tools and NFS share cleanup (high usage)
		Resolved		• Bstorm	T122508 Prevent overly-large log files
		Declined		None	T286847 Add webservice flag to mount project directory read-only
		Open		taavi	T127367 [toolforge,jobs-api,webservice,storage] Provide modern, non-NFS log solution for Toolforge tools
		Open		None	T400981 Check alerting for Loki
		Resolved		taavi	T386480 [o11y,logging,infra] Deploy Loki to store Toolforge tool log data
		Resolved		taavi	T396574 Provision object storage volumes for Loki
		Resolved		Andrew	T396594 Create OpenStack role that allows object storage access only
		Resolved		dcaro	T397339 Request creation of toolsbeta-logging VPS project
		Resolved		None	T397446 Request creation of tools-logging VPS project
		Resolved		Andrew	T398447 Increase tools-logging object storage quotas
		Open		None	T407852 [infra,logging] prometheus failing to fetch the metrics endpoint

Event Timeline

taavi created this task.Aug 1 2025, 9:13 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 1 2025, 9:13 AM

taavi added a subtask: T386480: [o11y,logging,infra] Deploy Loki to store Toolforge tool log data.Aug 1 2025, 9:36 AM

fnegri triaged this task as Medium priority.Aug 20 2025, 2:09 PM

taavi added a parent task: T407852: [infra,logging] prometheus failing to fetch the metrics endpoint.Tue, Jan 13, 5:42 PM

taavi removed a parent task: T407852: [infra,logging] prometheus failing to fetch the metrics endpoint.Tue, Jan 13, 5:44 PM

taavi added a subtask: T407852: [infra,logging] prometheus failing to fetch the metrics endpoint.