This means:
- drain and turn off cloudcephosd1029 (depool and destroy cookbook)
- kindly ask @VRiley-WMF to swap one of the drives for the new one
- do fio tests directly with that hard drive and one of the old ones at the same time (shaped like the current iops of the cluster, see https://docs.google.com/spreadsheets/d/1KJeexHRXOR6W2gkkqgnIOIxBBXnuylz8pNdE_3iyslo/edit?gid=0#gid=0 for the comparative numbers we need)
(copied from comment below) Did some tests, and we are on the clear, the new hard drives are performant enough (at low level) to handle the current load we have in the cluster:
Summary of improvement Needed improvement Difference (if green we are ok, if red we are not)
read iops 256.73% 183.33% 73.39%
read throughput 203.55% 183.33% 20.22%
write iops 377.22% 212.50% 164.72%
write throughput 277.84% 212.50% 65.34%
- add the cluster back to the node
- do some monitoring of the drive compared to the other ones (just in case)