Change Details

FQDN: `an-worker1172.eqiad.wmnet` Netbox: Set to `Failed` (https://netbox.wikimedia.org/dcim/devices/5037/) Priority: Medium #### Timeline - 2026-03-03 11:13 UTC — btullis ran `sre.hosts.reboot-single` (START) - 2026-03-03 12:08 UTC — cookbook returned END (PASS) after ~55 minutes, meaning the host came back up far enough to satisfy health checks - 2026-03-06 (or earlier) — host found SSH-unreachable; given the below backplane error, it prob very quickly fell back into failed state ``` Record: 26 Date/Time: 03/03/2026 11:17:38 Source: system Severity: Critical Description: The System Configuration Check operation resulted in the following issue: Comm Error: Backplane 0. ```

FQDN: `an-worker1172.eqiad.wmnet` Netbox: Set to `Failed` (https://netbox.wikimedia.org/dcim/devices/5037/) Priority: Medium **Host can be worked on at will** (it's already down) #### Timeline - 2026-03-03 11:13 UTC — btullis ran `sre.hosts.reboot-single` (START) - 2026-03-03 12:08 UTC — cookbook returned END (PASS) after ~55 minutes, meaning the host came back up far enough to satisfy health checks - 2026-03-06 (or earlier) — host found SSH-unreachable; given the below backplane error, it prob very quickly fell back into failed state ``` Record: 26 Date/Time: 03/03/2026 11:17:38 Source: system Severity: Critical Description: The System Configuration Check operation resulted in the following issue: Comm Error: Backplane 0. ```

FQDN: `an-worker1172.eqiad.wmnet` Netbox: Set to `Failed` (https://netbox.wikimedia.org/dcim/devices/5037/) Priority: Medium **Host can be worked on at will** (it's already down) #### Timeline - 2026-03-03 11:13 UTC — btullis ran `sre.hosts.reboot-single` (START) - 2026-03-03 12:08 UTC — cookbook returned END (PASS) after ~55 minutes, meaning the host came back up far enough to satisfy health checks - 2026-03-06 (or earlier) — host found SSH-unreachable; given the below backplane error, it prob very quickly fell back into failed state ``` Record: 26 Date/Time: 03/03/2026 11:17:38 Source: system Severity: Critical Description: The System Configuration Check operation resulted in the following issue: Comm Error: Backplane 0. ```