Got an alert of a downed datanode on analytics1032.
Datanode logs says:
org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 11, volumes configured: 12, volumes failed: 1, volume failures tolerated: 0
syslog says:
Sep 9 02:27:20 analytics1032 kernel: [3584610.310985] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK Sep 9 02:27:20 analytics1032 kernel: [3584610.310987] sd 0:2:4:0: [sde] CDB: ...