Page MenuHomePhabricator

mw2140.codfw.wmnet unreponsive, cannot be powercycled with serial console
Closed, ResolvedPublic

Description

Hey, @Papaul mw2140 seems dead and we cannot restart it, maybe you can do it manually?

Event Timeline

jcrespo created this task.May 25 2017, 5:41 PM
Restricted Application added a project: Operations. · View Herald TranscriptMay 25 2017, 5:41 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
RobH added a subscriber: RobH.May 25 2017, 6:16 PM

Once the powercycle is done, before the server is returned to service, full testing of the ilom needs to also happen.

IE: @Papaul, once you reset the ilom, make sure you can ssh into it and send powercycle commands.

Papaul claimed this task.May 26 2017, 4:07 PM
Papaul reassigned this task from Papaul to jcrespo.May 26 2017, 4:47 PM

Removed power for 5 minutes, Update IDRAC firmware from 1.57 to 2.32, update BIOS from 2.3 to 2.4.
System is back up.

Mentioned in SAL (#wikimedia-operations) [2017-05-29T07:01:48Z] <_joe_> reeanbling scap on mw2140, T166328

jcrespo reassigned this task from jcrespo to Joe.May 29 2017, 8:17 AM
jcrespo added a subscriber: Joe.

Ok to resolve, @Joe or is there any more debug we want to do?

Joe closed this task as Resolved.Jul 21 2017, 2:54 PM
Joe triaged this task as Medium priority.