Page MenuHomePhabricator

Write a cronjob to compress old Jenkins builds' logs
Closed, DeclinedPublic

Description

The console logs on gallium are a bit large. Jenkins knows how to read from them when they are compressed using gzip.

Under /var/lib/jenkins/jobs/ we should find any file named "log" which is more than a day old and gzip -9 them.

Details

Reference
bz63939

Related Objects

StatusSubtypeAssignedTask
DeclinedKrinkle

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:24 AM
bzimport set Reference to bz63939.
bzimport added a subscriber: Unknown Object (MLST).

Change 125991 had a related patch set uploaded by Hashar:
contint: compress Jenkins console logs once per day

https://gerrit.wikimedia.org/r/125991

Need to think about it, the tree traversal is a bit too long.

Change 125991 abandoned by Hashar:
contint: compress Jenkins console logs once per day

Reason:
Abandoning the change. The find command takes 5 hours to traverse the hierarchy and compress a few files. Meanwhile, that cause high CPU and I/O on the gallium server which has a bunch of interesting side effects such as: Jenkins dieing, jobs timeout etc.

Instead of compressing. I will just kill the job history after a few months.

https://gerrit.wikimedia.org/r/125991

Krinkle renamed this task from Jenkins: write a cronjob to compress old build history console logs to Write a cronjob to compress old Jenkins builds' logs.Jan 8 2015, 4:13 PM
Krinkle closed this task as Declined.
Krinkle claimed this task.
Krinkle updated the task description. (Show Details)
Krinkle set Security to None.
Krinkle removed a subscriber: Unknown Object (MLST).

Per Antoine, let's instead use Jenkins' built-in mechanism to purge old builds in general. Thus removing them entirely instead of compressing.

Krinkle added a parent task: Restricted Task.Mar 2 2015, 3:44 PM

I have installed Jenkins Compress Build Log Plugin which at least get the console output logs compressed :)