Page MenuHomePhabricator
Create Task
Maniphest T311526

es2033 crashed at Jun 28 ~15:34
Closed, ResolvedPublic
Actions

Description

Probably power related (guess because no shutdown logs- need to check hw logs). Disk/Raid many logs on reboot. Probably would need a data check on mysql re-start?

Related Objects

Event Timeline

jcrespo created this task.Jun 28 2022, 3:47 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 28 2022, 3:47 PM
jcrespo updated the task description. (Show Details)Jun 28 2022, 3:48 PM
Marostegui claimed this task.Jun 28 2022, 3:51 PM

Will check it!
Thanks for the initial triage!!

Marostegui moved this task from Triage to In progress on the DBA board.Jun 28 2022, 3:51 PM

Many icinga power alerts on IRC right now, so might be related

Volans subscribed.Jun 28 2022, 3:53 PM

Given that mc2032 also got rebooted around the same time and they are in the same rack it could be related to the work in T309956. It could also indicate that either they have the power cables not plugged correctly into the 2 different feeds or an issue with one of their power supply units.

Marostegui added a comment.Jun 28 2022, 3:54 PM

Yeah, definitely related to that maintenance

RhinosF1 subscribed.Jun 28 2022, 3:55 PM
Maintenance_bot added a project: SRE.Jun 28 2022, 4:29 PM
Marostegui added a comment.Jun 29 2022, 5:46 AM

Started a data check run

Marostegui closed this task as Resolved.Jun 30 2022, 8:22 AM

Data looks fine, resolving.

Maintenance_bot moved this task from In progress to Done on the DBA board.Jun 30 2022, 8:29 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits