The Zuul logs on gallium in /var/log/zuul are 4GB large. They are rotated by the python logging module but we should find out whether we can have them compressed as well to save up space when backing up.
Version: wmf-deployment
Severity: enhancement
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Restricted Task | |||||
| Declined | hashar | T65935 Zuul log should be compressed after rotation |
python logging does not compress them. They are rotated at midnight UTC, I guess we can trigger a gzip cron of /var/log/zuul/*.log.????-??-??
Compression looks fine to me:
root@integration-dev:/var/log/zuul# ll
total 224
-rw-rw-rw- 1 jenkins jenkins 9248 May 19 11:15 debug.log
-rw-rw-rw- 1 jenkins jenkins 113265 May 5 23:55 debug.log.2014-05-05.gz
-rw-rw-rw- 1 jenkins jenkins 1524 May 6 23:55 debug.log.2014-05-06.gz
-rw-rw-rw- 1 jenkins jenkins 1107 May 7 23:55 debug.log.2014-05-07.gz
-rw-rw-rw- 1 jenkins jenkins 1120 May 8 23:55 debug.log.2014-05-08.gz
-rw-rw-rw- 1 jenkins jenkins 1119 May 9 23:55 debug.log.2014-05-09.gz
-rw-rw-rw- 1 jenkins jenkins 1126 May 10 23:55 debug.log.2014-05-10.gz
-rw-rw-rw- 1 jenkins jenkins 1120 May 11 23:55 debug.log.2014-05-11.gz
-rw-rw-rw- 1 jenkins jenkins 1096 May 12 23:55 debug.log.2014-05-12.gz
-rw-rw-rw- 1 jenkins jenkins 1114 May 13 23:55 debug.log.2014-05-13.gz
-rw-rw-rw- 1 jenkins jenkins 1124 May 14 23:55 debug.log.2014-05-14.gz
-rw-rw-rw- 1 jenkins jenkins 1102 May 15 23:55 debug.log.2014-05-15.gz
-rw-rw-rw- 1 jenkins jenkins 1113 May 16 23:55 debug.log.2014-05-16.gz
-rw-rw-rw- 1 jenkins jenkins 1097 May 17 23:55 debug.log.2014-05-17.gz
-rw-rw-rw- 1 jenkins jenkins 1112 May 18 23:55 debug.log.2014-05-18.gz
-rw-rw-rw- 1 jenkins jenkins 0 May 5 11:58 gearman-server.log
-rw-rw-rw- 1 jenkins jenkins 28291 May 5 20:27 merger-debug.log
-rw-rw-rw- 1 jenkins jenkins 0 May 5 11:58 merger.log
-rw-rw-rw- 1 jenkins jenkins 143 May 6 20:40 zuul.log
-rw-rw-rw- 1 jenkins jenkins 10940 May 5 20:28 zuul.log.2014-05-05.gz
root@integration-dev:/var/log/zuul#
Change 127230 abandoned by Hashar:
zuul: compress log daily
Reason:
Will get a patch submitted upstream to have file compressed directly by Zuul
On review, both Ariel and Filippo asked to get the compression to be directly handled by Zuul. Will figure out a patch and send it upstream then upgrade our Zuul install.
There is enough disk space on the Zuul server for now.
Ops reject the idea to use a cron to zip them and I do not want to bother writing the python script to compress them.