Brotli is an open source data compression library based on a modern variant of the LZ77 algorithm, Huffman coding, and 2nd order context modeling. It typically gives an increase of 20% in compression density for text, with comparable compression and decompression speeds when compared to deflate.
We've begun experimenting with a compressor implementation for Cassandra, with promising results.
This will serve as a tracking issue; Sub-tasks to follow will cover evaluation, testing, and implementation and deployment (if necessary).
References:
- https://github.com/eevans/cassandra-brotli (Cassandra compressor)
- https://phabricator.wikimedia.org/T122028#1996542 (some initial test results)
- https://en.wikipedia.org/wiki/Brotli
- http://google-opensource.blogspot.com/2015/09/introducing-brotli-new-compression.html
- https://github.com/google/brotli
- https://github.com/MeteoGroup/jbrotli (Java bindings)
- https://tools.ietf.org/html/draft-alakuijala-brotli-01
- T93496: Improve revision compression in Cassandra / Brotli or LZMA support