Maniphest T320990

s6 master failure
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	RLazarus
	Oct 17 2022, 5:55 PM

Tags

Referenced Files

None

Subscribers

Description

Tracking task for followup associated with the 2022-10-15 incident in which the db1131, the s6 master, suddenly went offline, we suspect due to a bad DIMM. This rendered frwiki, jpwiki, ruwiki, and wikitech read-only for 24 minutes (plus 1-2 additional minutes during a master switchover). Incident report in progress, will post the link here shortly.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		RLazarus	T320990 s6 master failure
		Resolved		Ladsgroup	T320879 Switchover s6 master (db1131 -> db1173)
		Resolved		Jclark-ctr	T320994 Check DIMM A6 on db1131

Event Timeline

RLazarus triaged this task as Medium priority.Oct 17 2022, 5:55 PM

RLazarus created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 17 2022, 5:55 PM

RLazarus added a subtask: T320879: Switchover s6 master (db1131 -> db1173).Oct 17 2022, 5:56 PM

RhinosF1 subscribed.Oct 17 2022, 5:56 PM

Draft: https://wikitech.wikimedia.org/wiki/Incidents/2022-10-15_s6_master_failure

Jclark-ctr closed subtask T320994: Check DIMM A6 on db1131 as Resolved.Oct 18 2022, 3:53 PM

After DIMM replacement, it looks good now. I will slowly start repooling it now.

• Quiddity added a project: User-notice.Oct 20 2022, 7:49 PM

• Quiddity moved this task from To Triage to In current Tech/News draft on the User-notice board.

• Quiddity mentioned this in T320879: Switchover s6 master (db1131 -> db1173).

Great work everyone!

Now that we have the incident report, can this be closed?

Yup

• Quiddity moved this task from In current Tech/News draft to Already announced/Archive on the User-notice board.Oct 27 2022, 10:52 PM

Maintenance_bot edited projects, added User-notice-archive; removed User-notice.Nov 6 2022, 11:30 PM