Iridium (phabricator): Disk space is low
Closed, ResolvedPublic

Description

At 1am utc +0 there is a warning that iridium (phabricator) is low on space

[01:10:18] <icinga-wm> PROBLEM - Disk space on iridium is CRITICAL: DISK CRITICAL - free space: / 350 MB (3% inode=54%)

Paladox created this task.Jan 2 2017, 1:17 AM
Restricted Application added subscribers: TerraCodes, Aklapper. · View Herald TranscriptJan 2 2017, 1:17 AM
Paladox edited the task description. (Show Details)Jan 2 2017, 1:18 AM
Paladox changed the title from "Iridium: Disk space is low" to "Iridium (phabricator): Disk space is low".

I cleaned up some temp files, disk space is still very low.

Mentioned in SAL (#wikimedia-operations) [2017-01-03T16:49:41Z] <mutante> iridium (phab) - reduce process accounting from 30 days to 10 days to save disk space used by /var/log/account, run /etc/cron.daily/acct (T154407)

Mentioned in SAL (#wikimedia-operations) [2017-01-03T17:04:50Z] <mutante> iridium (phab) - apt-get clean ; find /var/log/account/ -mtime +10 -delete ; find /var/log/atop/ -mtime +10 -delete (T154407)

Dzahn closed this task as "Resolved".Jan 3 2017, 5:11 PM
Dzahn claimed this task.

back to > 3GB free ( ~ 64% use).

This should (also) be solved by making the root partition larger when doing T152129 and by stopping Phabricator from leaving old files in /tmp (T150396).