As observed when investigating T252898: echostore connection error in Beta Cluster, a transient Cassandra failure isn't (always) properly handled by the GoCQL driver.
May 20 15:37:33 deployment-echostore01 docker[10577]: {"msg":"Error reading from storage (gocql: no hosts available in the pool)","appname":"sessions","time":"2020-05-20T15:37:33Z","level":"ERROR","request_id":"00000000-0000-0000-0000-000000000000"}
May 20 15:37:33 deployment-echostore01 docker[10577]: {"msg":"Error reading from storage (gocql: no hosts available in the pool)","appname":"sessions","time":"2020-05-20T15:37:33Z","level":"ERROR","request_id":"00000000-0000-0000-0000-000000000000"}
May 20 15:37:33 deployment-echostore01 docker[10577]: {"msg":"Error reading from storage (gocql: no hosts available in the pool)","appname":"sessions","time":"2020-05-20T15:37:33Z","level":"ERROR","request_id":"00000000-0000-0000-0000-000000000000"}
May 20 15:37:42 deployment-echostore01 docker[10577]: {"msg":"Error reading from storage (gocql: no hosts available in the pool)","appname":"sessions","time":"2020-05-20T15:37:42Z","level":"ERROR","request_id":"00000000-0000-0000-0000-000000000000"}
May 20 15:37:42 deployment-echostore01 docker[10577]: {"msg":"Error reading from storage (gocql: no hosts available in the pool)","appname":"sessions","time":"2020-05-20T15:37:42Z","level":"ERROR","request_id":"00000000-0000-0000-0000-000000000000"}
May 20 15:41:29 deployment-echostore01 docker[10577]: {"msg":"Error writing to storage (gocql: no hosts available in the pool)","appname":"sessions","time":"2020-05-20T15:41:29Z","level":"ERROR","request_id":"00000000-0000-0000-0000-000000000000"}This seems to be: gocql/gocql/issues/915
gocql/gocql/issues/915#issuecomment-325329596 documents how to go about reproducing this, and Github user Zariel has requested that someone do so with the gocql_debug build tag enabled, and provide them with them the output (we should probably begin there).