Maniphest T196873

ms-be1036 in power off status, not responsive to power on commands
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	elukey
	Jun 11 2018, 9:06 AM

Project Tags

Referenced Files

None

Subscribers

Description

ms-be1036 went down this (EU) morning and it is currently in power off status. The last iLO log is from May, and I can't bring it on again using the power on/reset commands (via iLO).

Event Timeline

elukey created this task.Jun 11 2018, 9:06 AM

elukey triaged this task as Medium priority.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 11 2018, 9:06 AM

ms-be1036 will no power back manually either. I tried pulling the PSU"s out, waiting several minutes and all I get is a flashing green light on the power button. I am able to login to the ilo and download the AHS report (a requirement by HP).

I will submit a support ticket with HP.

• Cmjohnson moved this task from Backlog to High Priority Task on the ops-eqiad board.Jun 11 2018, 3:40 PM

Thanks @Cmjohnson ! Please treat this with urgency, do you know if there's an ETA? If more than a couple of days I'll remove the machine from swift.

@fgiunchedi I submitted a ticket with HP. I recommend removing the server from swift until it's fixed since I do not know what it's going to take to fix at this time.

@Cmjohnson ok! thanks, I'll being removing the machine from swift tomorrow

• Cmjohnson moved this task from High Priority Task to Blocked on the ops-eqiad board.Jun 12 2018, 5:20 PM

A new system board is required. I will coordinate with HP to get this taken care of ASAP.

Required part is

775400-001 System I/O board (motherboard) assembly - For Intel Xeon E5-2600 series v3 (Haswell) processors ONLY - Includes subpan, thermal grease, alcohol pad, and instruction card - Processors must be the same spare part number

Qty-1

Mentioned in SAL (#wikimedia-operations) [2018-06-13T07:33:19Z] <godog> start removing ms-be1036 from swift rings - T196873

Mentioned in SAL (#wikimedia-operations) [2018-06-15T09:20:46Z] <godog> fully remove ms-be1036 from swift due to hw failure - T196873

This is scheduled for this coming Friday 29/6/2018 at 1000(EST)

I pushed the schedule and the HP tech came today. The server is back online. @godog please resolve if satisfied.

• Vvjjkkii renamed this task from ms-be1036 in power off status, not responsive to power on commands to mabaaaaaaa.Jul 1 2018, 1:05 AM

• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).

• Vvjjkkii updated the task description. (Show Details)

• Vvjjkkii removed a subscriber: Aklapper.

elukey renamed this task from mabaaaaaaa to ms-be1036 in power off status, not responsive to power on commands.Jul 2 2018, 6:15 AM

elukey removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.

elukey updated the task description. (Show Details)

@Cmjohnson I can access the server but not its IPMI console, expected?

fgiunchedi added a project: User-fgiunchedi.Jul 4 2018, 7:46 AM

fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.

Mentioned in SAL (#wikimedia-operations) [2018-07-10T09:25:28Z] <godog> swift eqiad-prod add ms-be1036 back gradually - T196873

Mentioned in SAL (#wikimedia-operations) [2018-07-16T08:29:12Z] <godog> put back ms-be1036 to full weight - T196873

Host is back in service

ms-be1036 in power off status, not responsive to power on commandsClosed, ResolvedPublicActions

Description

Event Timeline

ms-be1036 in power off status, not responsive to power on commands
Closed, ResolvedPublic
Actions