Got this while diagnosing the UNKNOWN check for ms-be1016
root@ms-be1016:~# hpssacli controller all show Smart Array P840 in Slot 1 (sn: PDNNF0ARH7Z0QD) CACHE STATUS PROBLEM DETECTED: The cache on this controller has a problem. To prevent data loss, configuration changes to this controller are not allowed. Please replace the cache to be able to continue to configure this controller.
I suspect this needs physical investigation. While the data on the machine itself is redundant, if we plan any long (>= some days) then we should rebalance the cluster and migrate data off the machine itself.