Aug 23 17:00:00 webperf1004 arclamp-log.py[1919414]: /srv/deployment/performance/arc-lamp/arclamp-log.py:112: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used Aug 23 17:38:15 webperf1004 arclamp-log.py[2011266]: /srv/deployment/performance/arc-lamp/arclamp-log.py:112: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used Aug 23 18:28:15 webperf1004 arclamp-log.py[2413390]: /srv/deployment/performance/arc-lamp/arclamp-log.py:112: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
We should at least update the code to match the effective runtime behaviour, but also revisit whether it still makes sense. I recall that we intentionally picked line mode for the logic to work correctly, so it's possible that the current byte chunks are causing it to corrupt or skip some entries. If raw chunks are handled fine, then we can look at what a good chunk size would be here, e.g. trade-off waiting too long for input (keep producing hourly logs and flame graphs) vs processing large enough chunks to avoid back pressure.