Page MenuHomePhabricator

Zuul log should be compressed after rotation
Closed, DeclinedPublic

Description

The Zuul logs on gallium in /var/log/zuul are 4GB large. They are rotated by the python logging module but we should find out whether we can have them compressed as well to save up space when backing up.


Version: wmf-deployment
Severity: enhancement

Details

Reference
bz63935

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:24 AM
bzimport set Reference to bz63935.

python logging does not compress them. They are rotated at midnight UTC, I guess we can trigger a gzip cron of /var/log/zuul/*.log.????-??-??

Change 127230 had a related patch set uploaded by Hashar:
zuul: compress log daily

https://gerrit.wikimedia.org/r/127230

Being tested on integration-dev.eqiad.wmflabs instance which has Zuul.

Compression looks fine to me:

root@integration-dev:/var/log/zuul# ll
total 224
-rw-rw-rw- 1 jenkins jenkins 9248 May 19 11:15 debug.log
-rw-rw-rw- 1 jenkins jenkins 113265 May 5 23:55 debug.log.2014-05-05.gz
-rw-rw-rw- 1 jenkins jenkins 1524 May 6 23:55 debug.log.2014-05-06.gz
-rw-rw-rw- 1 jenkins jenkins 1107 May 7 23:55 debug.log.2014-05-07.gz
-rw-rw-rw- 1 jenkins jenkins 1120 May 8 23:55 debug.log.2014-05-08.gz
-rw-rw-rw- 1 jenkins jenkins 1119 May 9 23:55 debug.log.2014-05-09.gz
-rw-rw-rw- 1 jenkins jenkins 1126 May 10 23:55 debug.log.2014-05-10.gz
-rw-rw-rw- 1 jenkins jenkins 1120 May 11 23:55 debug.log.2014-05-11.gz
-rw-rw-rw- 1 jenkins jenkins 1096 May 12 23:55 debug.log.2014-05-12.gz
-rw-rw-rw- 1 jenkins jenkins 1114 May 13 23:55 debug.log.2014-05-13.gz
-rw-rw-rw- 1 jenkins jenkins 1124 May 14 23:55 debug.log.2014-05-14.gz
-rw-rw-rw- 1 jenkins jenkins 1102 May 15 23:55 debug.log.2014-05-15.gz
-rw-rw-rw- 1 jenkins jenkins 1113 May 16 23:55 debug.log.2014-05-16.gz
-rw-rw-rw- 1 jenkins jenkins 1097 May 17 23:55 debug.log.2014-05-17.gz
-rw-rw-rw- 1 jenkins jenkins 1112 May 18 23:55 debug.log.2014-05-18.gz
-rw-rw-rw- 1 jenkins jenkins 0 May 5 11:58 gearman-server.log
-rw-rw-rw- 1 jenkins jenkins 28291 May 5 20:27 merger-debug.log
-rw-rw-rw- 1 jenkins jenkins 0 May 5 11:58 merger.log
-rw-rw-rw- 1 jenkins jenkins 143 May 6 20:40 zuul.log
-rw-rw-rw- 1 jenkins jenkins 10940 May 5 20:28 zuul.log.2014-05-05.gz
root@integration-dev:/var/log/zuul#

Change 127230 abandoned by Hashar:
zuul: compress log daily

Reason:
Will get a patch submitted upstream to have file compressed directly by Zuul

https://gerrit.wikimedia.org/r/127230

On review, both Ariel and Filippo asked to get the compression to be directly handled by Zuul. Will figure out a patch and send it upstream then upgrade our Zuul install.

hashar lowered the priority of this task from Medium to Lowest.
hashar set Security to None.
Krinkle added a parent task: Restricted Task.Mar 2 2015, 3:44 PM
hashar claimed this task.

There is enough disk space on the Zuul server for now.

Ops reject the idea to use a cron to zip them and I do not want to bother writing the python script to compress them.