We got an alert today that we missed data in CRUX. When I tried to login to the server I couldn't access it. Looking at horizon it was up and running. I could access the WPT instance that also run at wiki cloud, so it was really strange. I tried to compare the verbose ssh log but couldn't understand what was going on.
I soft restarted the instance in Horizon and guess what, I could access the instance again. I've manually run the scripts to backfill data but we will miss data for one day.