Experiments in T122028: RFC: Chunked storage algorithms for archival data vs. large-window brotli compression showed very significant compression gains, as well as some performance gains when using Brotli compression with large compression block sizes.
To leverage this in production, the following steps are needed:
- Set up a larger (more realistic) staging cluster: T136340
- Load test brotli in staging.
- Deploy the brotli jar to production.
- Enable brotli compression for specific tables.
- Gradually ramp up the compression block size while keeping an eye on performance, stability and compression ratios.