Zstandard – Fast real-time compression algorithm
We have a delta-encoded binary format for trade logs that we use and zstd still does a pretty good job on them for the last bit.<p>It's just a pity that storage providers won't implement transparent compression. We store our logs on S3 and on-prem and it's just annoying that we have to store them compressed on S3. Would be cool to just store without the compression stage and have S3 do that bit.<p>I haven't tried btrfs with the transparent zstd compression, but perhaps worth a shot.