Page MenuHomePhabricator

Deploy Brotli compression support for Cassandra
Closed, ResolvedPublic

Description

Experiments in T122028: RFC: Chunked storage algorithms for archival data vs. large-window brotli compression showed very significant compression gains, as well as some performance gains when using Brotli compression with large compression block sizes.

To leverage this in production, the following steps are needed:

  • Set up a larger (more realistic) staging cluster: T136340
  • Load test brotli in staging.
  • Deploy the brotli jar to production.
  • Enable brotli compression for specific tables.
  • Gradually ramp up the compression block size while keeping an eye on performance, stability and compression ratios.

Event Timeline

The impetus for this work was optimal storage utilization for full document history. Since we are currently focused on storage of current revisions, (with long-term storage on hold) this doesn't seem relevant any more.