Question: More and more [`tar`](http://www.gnu.org/software/tar/manual/tar.html) archives use the [`xz`](http://en.wikipedia.org/wiki/Xz) format based on LZMA2 for compression instead of the traditional [`bzip2(bz2)`](http://en.wikipedia.org/wiki/Bz2) compression. In fact *kernel.org* made a late "*Good-bye bzip2*" [announcement, 27th Dec. 2013](https://www.kernel.org/happy-new-year-and-good-bye-bzip2.html), indicating kernel sources would from this point on be released in both tar.gz and tar.xz format - and on the main page of the [website](https://www.kernel.org/) what's directly offered is in `tar.xz`.

Are there any specific reasons explaining why this is happening and what is the relevance of [`gzip`](http://en.wikipedia.org/wiki/Gzip) in this context?

Answer: 
For distributing archives over the Internet, the following things are generally a priority:

1. Compression ratio (i.e., how small the compressor makes the data);
2. Decompression time (CPU requirements);
3. Decompression memory requirements; and
4. Compatibility (how wide-spread the decompression program is)

Compression memory & CPU requirements aren't very important, because you can use a large fast machine for that, and you only have to do it once.

Compared to bzip2, xz has a better compression ratio and lower (better) decompression time. It, however—at the compression settings typically used—requires more memory to decompress[1] and is somewhat less widespread. Gzip uses less memory than either.

So, both gzip and xz format archives are posted, allowing you to pick:

* Need to decompress on a machine with *very* limited memory (<32 MB): gzip. Given, not very likely when talking about kernel sources.
* Need to decompress minimal tools available: gzip
* Want to save download time and/or bandwidth: xz

There isn't really a realistic combination of factors that'd get you to pick bzip2. So its being phased out.

I looked at compression comparisons in [a blog post](http://stephane.lesimple.fr/blog/2010-07-20/lzop-vs-compress-vs-gzip-vs-bzip2-vs-lzma-vs-lzma2xz-benchmark-reloaded.html). I didn't attempt to replicate the results, and I suspect some of it has changed (mostly, I expect `xz` has improved, as its the newest.)

(There are some specific scenarios where a good bzip2 implementation may be preferable to xz: bzip2 can compresses a file with lots of zeros and genome DNA sequences better than xz. Newer versions of xz now have an (optional) block mode which allows data recovery after the point of corruption and parallel compression and [in theory] decompression. Previously, only bzip2 offered these.[2] However none of these are relevant for kernel distribution)

---

1: In archive size, `xz -3` is around `bzip -9`. Then xz uses less memory to decompress. But `xz -9` (as, e.g., used for Linux kernel tarballs) uses much more than `bzip -9`. (And even `xz -0` needs more than `gzip -9`).

2: [F21 System Wide Change: lbzip2 as default bzip2 implementation](http://markmail.org/message/226x5nlbigycxqzb)
LZMA2 is a block compression system whereas gzip is not. This means that LZMA2 lends itself to multi-threading. Also, if corruption occurs in an archive, you can generally recover data from subsequent blocks with LZMA2 but you cannot do this with gzip. In practice, you lose the entire archive with gzip subsequent to the corrupted block. With an LZMA2 archive, you only lose the file(s) affected by the corrupted block(s). This can be important in larger archives with multiple files.