Page MenuHomePhabricator

daily cronspam from elastic100* hosts
Closed, ResolvedPublic

Description

Hi!

Daily root@ cronspam from the elastic100* hosts:

Cron Daemon root@elastic1006.eqiad.wmnet via wikimedia.org

to root
/etc/cron.daily/logrotate:
error: error creating output file /var/log/exim4/mainlog.1.gz: File exists
error: error creating output file /var/log/syslog.1.gz: File exists
run-parts: /etc/cron.daily/logrotate exited with return code 1

Any attempt to fix this would be awesome :)

Thanks!

Luca

Event Timeline

Thanks for the report. @Gehel should take a look at this when he gets the chance, to see if there's anything he can do to fix it.

It seems that this happens because of empty gzipped log files (size == 0):

sudo find /var/log -name "*.gz" -size 0

I can find those empty .gz files on the following nodes:

elastic1006.eqiad.wmnet: /var/log/auth.log.1.gz
elastic1006.eqiad.wmnet: /var/log/exim4/mainlog.1.gz
elastic1012.eqiad.wmnet: /var/log/exim4/mainlog.1.gz
elastic1012.eqiad.wmnet: /var/log/auth.log.1.gz
elastic1012.eqiad.wmnet: /var/log/syslog.1.gz
elastic1016.eqiad.wmnet: /var/log/kern.log.1.gz
elastic1016.eqiad.wmnet: /var/log/exim4/mainlog.1.gz
elastic1016.eqiad.wmnet: /var/log/auth.log.1.gz
elastic1016.eqiad.wmnet: /var/log/syslog.1.gz
elastic1026.eqiad.wmnet: /var/log/exim4/mainlog.1.gz
elastic1026.eqiad.wmnet: /var/log/syslog.1.gz
elastic1026.eqiad.wmnet: /var/log/upstart/systemd-logind.log.7.gz
(no issue on codfw)

Cleanup done, will investigate more if the issue arise again.