Why Not ZFS (2021)
This article is full of misinformation to the point that I&#x27;m not sure there&#x27;s many redeeming points in it, and if there are, they&#x27;re drowned out by wrong information:<p>&gt; Out-of-tree and will never be mainlined<p>So the article is heavily Linux-focused. Fine. To an end user, having a driver that&#x27;s not part of the mainline kernel sources is hardly a deal breaker. It still ends up running in kernel space with all the advantages that comes with.<p>&gt;&gt; Ubuntu ships ZFS as part of the kernel, not even as a separate loadable module. This redistribution of a combined CDDL&#x2F;GPLv2 work is probably illegal.<p>Canonical lawyers have obviously disagreed with this assessment. So far so good.<p>&gt;&gt; Red Hat will not touch this with a bargepole.<p>Red Hat also doesn&#x27;t touch btrfs. Or basically anything that&#x27;s not ext4 and XFS.<p>&gt;&gt; You could consider trying the fuse ZFS instead of the in-kernel one at least, as a userspace program it is definitely not a combined work.<p>No, you really should not. zfs-fuse has not been maintained in over a decade, doesn&#x27;t even remotely come close to supporting the features of modern ZFS, and frankly... it&#x27;s FUSE. It&#x27;s slow as molasses.<p>&gt; Slow performance of encryption<p>&gt;&gt; ZoL did workaround the Linux symbol issue above by disabling all use of SIMD for encryption, reducing the performance versus an in-tree filesystem.<p>Only partially true, but the damage is limited to some metadata structures and the bulk of encryption code does use SIMD instructions (eg, the parts that encrypt your file data).<p>&gt; Rigid<p>&gt;&gt; This RAID-X0 (stripe of mirrors) structure is rigid, you can’t do 0X (mirror of stripes) instead at all. You can’t stack vdevs in any other configuration.<p>Hard to say why it&#x27;s useful. Hardly anyone ever chooses that configuration even in the non-ZFS world. It&#x27;s the same capacity tradeoff for a different approach.<p>&gt;&gt; For argument’s sake, let’s assume most small installations would have a pool with only a single RAID-Z2 vdev.<p>Okay, not an item that can actually be refuted, but the idea that &quot;most small installations&quot; have only a single raidz2 vdev is a stretch. I&#x27;d wager there&#x27;s a heck of a lot more single-disk and two-disk mirror configurations than all the other types combined.<p>&gt; Can’t add&#x2F;remove disks to a RAID<p>Everything here is accurate. Part of it is because of ZFS&#x27;s original target audience, part of it is really hard math problems that hadn&#x27;t been solved in a manner where you can actually pull it off before the death of the universe. Mind that mdadm isn&#x27;t exactly magic either -- it&#x27;ll frequently refuse some operations to reshape a RAID array, and it&#x27;s not quite as upfront in the documentation about what those scenarios are.<p>&gt; RAIDZ is slow<p>raidz has to make sure all the disks have the same set of data synchronized. This is a major reason people are generally recommended to use mirrors instead of raidz.<p>&gt; File-based RAID is slow<p>ZFS does not use file-based RAID, it uses block-based RAID. Yes, it knows what blocks are used and will only need to scrub&#x2F;resilver those blocks.<p>&gt;&gt; Sequential read&#x2F;write is a far more performant workload for both HDDs and SSDs.<p>Yes it is. Which is why ZFS 2.0 introduced sequential scrubs.<p>&gt;&gt; It’s especially bad if you have a lot of small files.<p>ZFS isn&#x27;t file-based, it doesn&#x27;t matter if you have a single 2TB file or a million files. It&#x27;s the same work either way.<p>&gt; Real-world performance is slow<p>Comparing to ext4 isn&#x27;t the most fair thing to do. ext4 is a dumb file system that will give you raw disk performance, every time. If this is the utmost priority, use ext4. ZFS adds compression, checksums, and redundancy to the mix. Extra protection means it&#x27;s a bit slower.<p>&gt; Performance degrades faster with low free space<p>&gt;&gt; It’s recommended to keep a ZFS volume below 80 - 85% usage and even on SSDs. This means you have to buy bigger drives to get the same usable size compared to other filesystems.<p>Basically every file system gets bad when in high-utilization, and probably should be taken as a sign to either upgrade the storage or start deleting.<p>The threshold for where ZFS starts getting painful depends on which anecdote you listen to. I&#x27;ve heard from people running up to 95% utilization on an SSD without feeling the burn.<p>&gt;&gt; ZFS’s problem is on an entirely different level because it does not have a free-blocks bitmap at all.<p>ZFS has had a spacemap feature since 0.6.4 and an improved version since 0.8.0. If the output of the &quot;zpool list&quot; command has a FRAG value other than a hyphen, your pool is using this feature already.<p>&gt; Layering violation of volume management<p>Basically &quot;but muh Unix philosophy&quot; argument. It&#x27;s tiresome :)<p>In order for ZFS to do what it does so well, it has to incorporate all these features that used to be in different layers.  The reason that resilvering a ZFS pool is so much faster than the whole-disk RAID solutions of yore? Precisely because it knows exactly what blocks are in-use and what are not.<p>&gt;&gt; If you use ZFS’s volume management, you can’t have it manage your other drives using ext4, xfs, UFS, ntfs filesystems.<p>You can create volumes and store ext4, xfs, ufs, ntfs filesystems on top of ZFS just fine.<p>&gt;&gt; And likewise you can’t use ZFS’s filesystem with any other volume manager.<p>You <i>can</i>, but you probably shouldn&#x27;t.<p>&gt; Doesn’t support reflink<p>Some work in progress is around to do it, but not in a release version yet. Doesn&#x27;t mean it never will have it.<p>&gt; High memory requirements for dedupe<p>Indeed, and there are research projects at a new dedup algorithm to drastically reduce the memory requirement. Don&#x27;t use dedup unless you really really need it.<p>&gt; Dedupe is synchronous instead of asynchronous<p>Dedup is meant to work on live data so it does what you want immediately.<p>&gt;&gt; (By comparison, btrfs’s deduplication and Windows Server deduplication run as a background process, to reclaim space at off-peak times.)<p>Sometimes &quot;off-peak times&quot; don&#x27;t exist, and this just highlights the limitations of the two mentioned technologies: they don&#x27;t have an online dedup mode. I know at least for btrfs, you have to completely take the file system offline to do a dedup pass after-the-fact (and the end result is the same as the aforementioned reflink).<p>&gt; High memory requirements for ARC<p>Basically a long rundown of Linux&#x27;s own page cache fighting with the ARC. It&#x27;s actually a fair point, but it&#x27;s probably overblown. It&#x27;s nowhere as dire as the section makes it out to be.<p>&gt;&gt; Even on FreeBSD where ZFS is supposedly better integrated, ZoL still pretends that every OS is Solaris via the Solaris Porting Layer (SPL) and doesn’t use their page cache neither. This design decision makes it a bad citizen on every OS.<p>FreeBSD&#x27;s technical implementation of how they ported ZFS doesn&#x27;t really matter, and this is the second time the article has said &quot;supposedly better integrated&quot; -- it&#x27;s not supposed, it&#x27;s literally as well-integrated as ZFS on Solaris is. (Guess what? Solaris still has UFS too, it&#x27;s pretty much on-par with FreeBSD)<p>&gt; Buggy<p>The flimsiest argument of them all :)<p>Bug trackers track bugs. Some of them aren&#x27;t even bugs (as the nature of user-submitted bug reports are). It goes more to show the popularity and widespread use of ZFS than anything else.<p>I&#x27;d be far more concerned about a software project that has no bug reports on display at all.<p>&gt; No disk checking tool (fsck)<p>&gt;&gt; Yikes.<p>There is, it&#x27;s called scrubbing. See &quot;man zpool-scrub&quot; for details.<p>&gt;&gt; In ZFS you can use zpool clear to roll back to the last good snapshot, that’s better than nothing.<p>&quot;zpool clear&quot; is an administrative command to wipe away error reports from storage devices. It should only be used when an administrator determines that the problem is not a bad disk.<p>ZFS pools can be made of many file systems with any snapshots you desire. There is no &quot;the last good snapshot&quot;. Maybe zpool checkpoints are what the author is thinking about, but I doubt it.<p>&gt;&gt; merely rolling back to the last good snapshot as above does not verify the deduplication table (DDT) and this will cause all snapshots to be unmountable<p>That really should be impossible. I&#x27;ve never even heard of such a thing happening.<p>&gt;&gt; coupled with the above point (“Buggy”) if ZFS writes bad data to the disk or writes bad metaslabs, this is a showstopper<p>This is an error that is detected and provided as part of the &quot;zpool status&quot; command (and as mentioned, &quot;zpool clear&quot; can even clear the errors).<p>&gt;&gt; and so it should have an fsck.zfs tool that does more repair steps than just exit 0.<p>You could replace it with one that does a scrub, but that can take weeks on some pools :)<p>&gt; Things to use instead<p>&gt;&gt; The baseline comparison should just be ext4. Maybe on mdadm.<p>If you think mdadm+ext4 is comparable to ZFS, you are waaaaay off. Even btrfs can&#x27;t hold a candle to ZFS and it comes closer than mdadm+ext4.<p>Here in this section, the author comes across the term &quot;scrubbing&quot; but doesn&#x27;t really apply it in the way ZFS uses it.<p>&gt;&gt; Compression is usually not worthwhile<p>Hard disagree :)<p>&gt;&gt; Checksumming is usually not worthwhile<p>Disks lie. All the time. The claims of &quot;physical disk already has CRC checksums&quot; was around over 20 years ago when the ZFS project started, and the fact that disks lie or do not do strong protection is a huge reason that ZFS was created in the first place. The problem in 2022 remains the same as it was in 2000.<p>&gt; Summary<p>&gt;&gt; you can achieve all the same nice advanced features<p>You really can&#x27;t. It&#x27;s not even close. ZFS is so far ahead of the game, that even if some alternatives (eg, btrfs) offer a few similar features, they don&#x27;t even approach it.<p>&gt;&gt; ZFS also has a lot of tuning parameters to set.<p>Having tuning parameters and requiring them are two different things.<p>&gt;&gt; In the future we’re waiting to see what stratis<p>stratis is a dead-on-arrival joke.<p>&gt;&gt; bcachefs<p>Probably the only thing that has a shot at competing with ZFS.