Vanadium's / partition is full from logs in /var/log/upstart, of the form 'eventlogging_processor-client-side-events.log.1' etc. A lot of them seem to be large notices of events failing to validate?
I am temporarily moving the biggest of the files (51G eventlogging_processor-client-side-events.log.1) on to /srv. Someone who knows more about the EventLogging service should take a look.
Sorry I did not send e-mail to team yesterday. The root cause is a huge volume of events that are invalid that we cannot deal with.
Volume needs to be lower and invalid events need to be fixed.
Filesystem Size Used Avail Use% Mounted on /dev/md1 111G 60G 45G 58% / udev 3.9G 4.0K 3.9G 1% /dev tmpfs 798M 328K 798M 1% /run none 5.0M 0 5.0M 0% /run/lock none 3.9G 0 3.9G 0% /run/shm /dev/mapper/vg0-lv0 352G 104G 249G 30% /srv
^ looks resolved now?