db1101, one of the new hosts we recently bought just crashed
I will update this once I get more information.
The server was executing some heavy ALTER tables.
From the logs:
/admin1/system1/logs1/log1-> show record1 properties CreationTimestamp = 20171017114511.000000-300 ElementName = System Event Log Entry RecordData = Multi-bit memory errors detected on a memory device at location(s) DIMM_A4. RecordFormat = string Description RecordID = 5 /admin1/system1/logs1/log1-> show record3 properties CreationTimestamp = 20171012141437.000000-300 ElementName = System Event Log Entry RecordData = Correctable memory error rate exceeded for DIMM_A4. RecordFormat = string Description RecordID = 3