• - Provide FQDN of system.
• - If other than a hard drive issue, please depool the machine (and confirm that it’s been depooled) for us to work on it. If not, please provide time frame for us to take the machine down.
• - Put system into a failed state in Netbox.
• - Provide urgency of request, along with justification (redundancy, dependencies, etc)
• - Describe issue and/or attach hardware failure log. (Refer to https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook if you need help)
• - Assign correct project tag and appropriate owner (based on above). Also, please ensure the service owners of the host(s) are added as subscribers to provide any additional input.

• FQDN: rdb1014.eqiad.wmnet
• Urgency: High, only replica for redis-misc-pair1
• Issue: CPU 2 machine check error. Host went down during the week-end due to this error, tried to powercycle, it comes up for a little while then crashes again with the same error

-------------------------------------------------------------------------------
Record:      137
Date/Time:   07/22/2024 13:15:52
Source:      system
Severity:    Critical
Description: CPU 2 machine check error detected.
-------------------------------------------------------------------------------

Host not put into failed status due to T336275: Upgrade Netbox to 4.x in progress