The Reliability of Optical Disks
Aside from the author dismissing tape because &quot;drives are expensive&quot; and then concluding that the primary disadvantage to optical is &quot;the discs are expensive&quot; (wow, big surprise...) and then he doesn&#x27;t bother to actually calculate the costs of either...<p>...and not describing how they tested for errors...<p>...and not having particularly rigorous testing methodology (the DVD was apparently left for three months before he got around to checking it)...<p>...no organization looks at a requirement like &quot;archive these records for 50 years&quot; and then commands its IT department to only store the data on media that will be expected to survive 50 years.<p>What you do is store the data in such a way that your chance of losing data per year is within acceptable margins, on the most suitable storage method at the time, <i>and migrated periodically when that system&#x27;s chance of data loss falls below acceptable margins, appears to be getting too impractical to operate&#x2F;maintain, or there are improvements in reliability or practicality.</i><p>That can include factors like cost, availability of knowledgeable labor (ask the IRS how finding COBOL programmers is going), parts&#x2F;service availability&#x2F;cost, and so on.<p>Other comments:<p>&gt; HDDs, especially NAS disks, have an “always on” assumption<p>Even in the early 2000&#x27;s, commercial NASes were offering systems that could power down portions of the array that were unused. There was significant interest in minimizing opex, and that&#x27;s mostly power. Nowadays unraid and other solutions allow for the same.<p>&gt; And if you ever wanted to take disks offline and store them on a shelf, you don’t really know how long they’ll survive - unless you plug them in every now and then.<p>&quot;you don&#x27;t know if that HDD is gonna work if it&#x27;s sitting on a shelf&quot; is irrelevant <i>because your backups shouldn&#x27;t be sitting on a shelf for any significant period of time, regardless of the media&#x2F;mechanism</i>. See discussion above: you should be rotating your media off-site periodically, bringing it back for tests. The biggest problem with HDDs is that SATA connectors are not rated for frequent use; their connect&#x2F;disconnect rating is often in the range of &quot;hundred or so.&quot; (This can be partially addressed by using a cable that stays plugged into the drive, and replacing the cable when it is past a certain number of cycles.)<p>&gt;  if you ever wanted to take disks offline and store them on a shelf, you don’t really know how long they’ll survive - unless you plug them in every now and then.<p>&gt; And yet, the expectation is, that you will be able to read the disks without problem - even with zero maintenance. Indeed, that was the outcome of my ~15 year experiment!<p>That&#x27;s not the expectation, at all? CD-Rs used to have a lifetime measured in months before they start showing failures, especially if they had an adhesive label. I don&#x27;t understand how the author possibly could be reading ten year old CD-Rs. That&#x27;s basically impossible.<p>DVD-Rs faired a bit better, and bluray disks even better.<p>The issue with HDDs is the lubrication of the motor<p>&gt; any NAS will automatically check for errors, and notify if problems are found.<p>No. No. No. NO. This is probably one of the biggest myths of RAID and NAS devices.<p>Just because you have a RAID array doesn&#x27;t mean it is configured to scrub its arrays; this often has to be enabled.<p>Just because you have a RAID array scrubbing doesn&#x27;t mean you&#x27;re going to find out about it. Make sure your reporting works, and ideally report on success, not just failure, and have something else that reports on failure to receive<p>Last but not least:<p>Just because your RAID array controller (or software) finds a parity error or unreadable block doesn&#x27;t mean it does what you expect, and most people expect &quot;beep bop boop, parity&#x2F;mirror inconsistency detected. Fixed, meatsack! Pat yourself on the back for being smart in using RAID.&quot;<p>Reality: &quot;beep bop boop, parity or mirror inconsistency detected, so go and verify your files or restore from backup, meatsack.&quot;<p>RAID arrays cannot self-repair mirror or parity errors <i>because there is no way to do so.</i> The array controller has no idea whether it&#x27;s the parity bit that is wrong, or the data. In a mirror, it has no idea which drive is correct - just that one of them is bad. <i>RAID is not for data consistency.</i><p>This is why ZFS is usually the top pick for data consistency: it knows the checksum of files and metadata, in addition to the on-disk redundancy via mirrored copies or parity. If it finds an error, it can play out both scenarios to see which results in files or metadata that match their checksum.