Smaller and faster data compression with Zstandard
The modern trend of compressors is to use more memory to achieve speed. This is good if you're using big-iron cloud computers...<p><i>"Zstandard has no inherent limit and can address terabytes of memory (although it rarely does). For example, the lower of the 22 levels use 1 MB or less. For compatibility with a broad range of receiving systems, where memory may be limited, it is recommended to limit memory usage to 8 MB. This is a tuning recommendation, though, not a compression format limitation."</i><p><i>8MB</i> for the smallest preset? Back in the mid-2000s, I was attending a Jabber/XMPP discussion, about the viability of using libz for compressing the stream. It turned out that even just a <i>32kb</i> window is <i>huge</i> when your connection server is handling thousands of connections at a time, and they were investigating the effect of using a modified libz with an even  smaller window (it was hard-coded, back then).<p>I know Moore's law is in ZStandard's favor w.r.t. memory usage (what's 8MB when your server's got 64GB or more?), but I think it's useful to note that this is squarely aimed at web traffic backed by beefy servers.