On wtp1043, we have an alert related to a SMART failure, but upon inspection, the SMART checks pass if we `-T permissive` option:
```
wtp1043:~# smartctl -H /dev/sda
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.0-6-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
Short INQUIRY response, skip product id
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.
```
```
wtp1043:~# smartctl -T permissive -H /dev/sda
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.0-6-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
Short INQUIRY response, skip product id
=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK
```
... but now I see errors in the kernel logs trying to write to sda. The disk needs to be swapped out with the server powered off. No reimaging is needed, just:
- poweroff
- swap the disk
- poweron
- handoff to services-ops for validation and repooling.