mw2140.codfw.wmnet unreponsive, cannot be powercycled with serial console
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	jcrespo
	May 25 2017, 5:41 PM

Description

Hey, @Papaul mw2140 seems dead and we cannot restart it, maybe you can do it manually?

Restricted Application added a project: SRE. · View Herald TranscriptMay 25 2017, 5:41 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Once the powercycle is done, before the server is returned to service, full testing of the ilom needs to also happen.

IE: @Papaul, once you reset the ilom, make sure you can ssh into it and send powercycle commands.

Removed power for 5 minutes, Update IDRAC firmware from 1.57 to 2.32, update BIOS from 2.3 to 2.4.
System is back up.

Mentioned in SAL (#wikimedia-operations) [2017-05-29T07:01:48Z] <_joe_> reeanbling scap on mw2140, T166328

Ok to resolve, @Joe or is there any more debug we want to do?

Joe closed this task as Resolved.Jul 21 2017, 2:54 PM

Joe triaged this task as Medium priority.