Maniphest T398856

Write incident report for cirrussearch outage 2025-07-07 17:10-17:40 UTC
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	bking
	Jul 7 2025, 5:50 PM

Tags

Referenced Files

None

Subscribers

Description

Incident report here

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		bking	T398856 Write incident report for cirrussearch outage 2025-07-07 17:10-17:40 UTC
		Open		None	T398866 Warn user about cluster replication settings/LB pool status after elasticsearch rolling-operation cookbook fails; alert for cluster replication settings
		Resolved		bking	T400160 Investigate eqiad cluster quorum failure issues

Event Timeline

bking created this task.Jul 7 2025, 5:50 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 7 2025, 5:50 PM

bking updated the task description. (Show Details)Jul 7 2025, 9:31 PM

Aklapper added a project: Sustainability (Incident Followup).Jul 7 2025, 10:41 PM

bking updated the task description. (Show Details)Jul 7 2025, 10:46 PM

bking changed the task status from Open to In Progress.Jul 9 2025, 3:39 PM

bking claimed this task.

bking triaged this task as Medium priority.

bking edited projects, added Data-Platform-SRE (2025.07.05 - 2025.07.25); removed Data-Platform-SRE.

bking moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2025.07.05 - 2025.07.25) board.Jul 9 2025, 4:52 PM

TJones removed a project: Discovery-Search.Jul 14 2025, 3:21 PM

bking mentioned this in T400160: Investigate eqiad cluster quorum failure issues.Jul 24 2025, 7:11 PM

BTullis edited projects, added Data-Platform-SRE (2025.07.26 - 2025.08.15); removed Data-Platform-SRE (2025.07.05 - 2025.07.25).Jul 28 2025, 4:47 PM

BTullis moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2025.07.26 - 2025.08.15) board.

bking changed the status of subtask T400160: Investigate eqiad cluster quorum failure issues from Open to In Progress.Aug 19 2025, 6:02 PM

The incident report is finished, so I'm closing out this ticket. The investigation continues in T400160...

BTullis moved this task from Done to Reported on the Data-Platform-SRE (2025.07.26 - 2025.08.15) board.Aug 20 2025, 10:32 AM

bking closed subtask T400160: Investigate eqiad cluster quorum failure issues as Resolved.Aug 21 2025, 10:10 PM