This would fit nicely with the upcoming refactor of Arc Lamp (see T200109, T200108).
This task is a follow-up to the webperf*002 hardware alert from T235425.
During the alert response we (accidentally) found that compressing the oldest 30 days of the 45 days of data in arclamp/logs/daily, reduced it from 75G to 45G. (We normally store 90 days worth there, but I already deleted the older half before measuring).
Supporting this natively (in a way that doesn't break our SVG generator/garbage-collector) would buy us some time and might allow us to keep the current retention span at least until traffic grows and/or util/if we migrate XHGui's db to the same server (T180761).
See also T235455: Resolve arclamp disk exhaustion problem (Oct 2019).
Sub tasks:
- Support compressed log files in internal housekeeping and file readers readers (arclamp-grep, arclamp-svg, and log gargage collection).
-
Serve compresssed files on the web server in a way that browsers and most programmatic clients will still naturally decode. Or only compress fiiles that are more than a few days old - Regularly compress log files (perhaps only for "finished", older, files).
