Answers.Body: <p>We have been using a Synology multibay NAS diskstation similar to this <a href="https://www.synology.com/en-us/products/DS418" rel="noreferrer">one</a>. Additional expansion units such as <a href="https://www.synology.com/en-us/products/DX517" rel="noreferrer">this</a> can be added to daisy-chain to larger storage. Access to drives can be direct, via network, or you can also have an online access. Requires NAS drives (more expensive than normal internal hard drives). Over the years we have had several drives go bad-- our IT is alerted &amp; they are able to change out the drives without any loss of data.</p>
  <p>In comparison, we have lost data in less than a year on other external hard drive storage devices (unlucky!).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-28T18:24:39.560Z
Answers.Text: I, too, use the Synology NAS. I believe in redundancy, so I also set up cloud storage (through my institution) for my files.  But the NAS is where I put all my data first.
Answers.UserDisplayName: Laura Kloepper
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-20T22:32:34.980Z
Answers.LastActivityDate: 2022-07-20T22:32:34.980Z
Answers.UserDisplayName: Shannon
Answers.Body: <p>I have many hard drives and a backup hard drive for each of those, and for many of them, my collaborators have another copy.  Hard drives have failed but the backups have saved the data.</p>
  <p>Another system like the Synology system is the Nexsan Beast which can hold up to 1 PB of data and has the option for expansion drives as well.  It can also have up to two drives fail without any data loss, and those drives can be replaced as needed. It’s not inexpensive though and my understanding is it has a lifetime of 5-10 years.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-20T23:08:33.677Z
Answers.LastActivityDate: 2022-07-20T23:08:33.677Z
Answers.UserDisplayName: Melissa 
Answers.Body: <p>Someone said, the only reliable long-term storages of information are petroglyphs.</p>
  <p>I see three aspects:</p>
  <ul>
  <li>media change over time and modern computers have even no proper interfaces anymore;</li>
  <li>media size changes inviting recording of lots of noise (i.e. data without any information content);</li>
  <li>media degrade over time (even petroglyphs).</li>
  </ul>
  <p>My suggestions would be:</p>
  <ul>
  <li>when new more capable media emerge, spend some time in copying all relevant but only relevant old information to new media;</li>
  <li>remove all data from your storage (declutter your storage) that is older than, say 10 years and nobody has looked into.</li>
  </ul>
  <p>AFAIK, continuous copying is the only way to fight physical degradation of media (not only tapes), and one can only copy so much data during a day.</p>
  <p>The 2nd suggestion sounds dramatic, but if there is no interest of your data then why keep it?</p>
  <p>OK, I know, my noise may be information for someone else, but that sounds very often only like an excuse to not to declutter your data store.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-22T14:00:25.977Z
Answers.Text: Seconding the suggestion that getting rid of things that are unused is important! Assessing whether the issue is 'no one wants this' or 'no one can find this' is significant, but that is why collective archive projects are getting traction. There is only so much traffic that can result from 'contact the author' lines in publications
Answers.UserDisplayName: Sarah Vela
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-21T07:16:13.060Z
Answers.LastActivityDate: 2022-10-15T16:15:04.447Z
Answers.LastEditDate: 2022-10-15T16:15:04.447Z
Answers.UserDisplayName: WMXZ
Answers.Body: <p>Thus far, I'm primarily using open public data archives as my way to preserve &gt;5 years, since I know that most &quot;local&quot; solutions will fail or get lost. Openness means that <em>&quot;lots of copies keeps stuff safe&quot;</em>, as the saying goes. Good examples would be <strong>Zenodo</strong>, <strong>OSF</strong>, <strong>Dataverse</strong>, <strong>Archive.org</strong>.</p>
  <p>However, an issue with audio data (and other media) is size. Larger audio datasets can be &gt;1TB and public services may be unwilling to provide for such sizes. I don't currently have a general answer to offer for that.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-27T23:34:33.370Z
Answers.Text: If you are part of NOAA, the NODD is a resource that now supports data archival of large audio collections as active cloud storage on GCP. The caveat is that it must be up to publisher standards, publicly available, and relatively static. While it's still in it's fledgling stages it is available. If interested contact me or your science center IT reps.
Answers.UserDisplayName: Dan Woodrich
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-21T16:02:33.070Z
Answers.LastActivityDate: 2022-07-21T16:02:33.070Z
Answers.UserDisplayName: Dan Stowell
Answers.Body: <p>Using NAS has also worked for us. It's relatively cheap (a 20 TB hardrive is ~500$), easy to set up, easy to use. You can set your NAS diskstation up in a way that if one or two of the drives fails your data is not lost.</p>
  <p>I would still be much happier if we had another longer term archiving option available to us. I have thought about <strong>magnetic tape for data archiving</strong>. Their lifespan should be ~ <a href="https://www.techtarget.com/searchdatabackup/tip/How-to-estimate-the-lifespan-of-LTO-tapes" rel="nofollow noreferrer">30 years</a> . The cost of the tapes themselves seems quite low, but the cost of the tape drives are a bit too much for us at the moment.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-22T06:11:44.793Z
Answers.LastActivityDate: 2022-10-15T16:23:51.830Z
Answers.LastEditDate: 2022-10-15T16:23:51.830Z
Answers.UserDisplayName: Mirko Mustonen
Answers.Body: <p>For long-term storage, how about Blu-ray. Here's a sales pitch from Panasonic:</p>
  <blockquote>
  <p>Panasonic Blu-ray™ discs have an archival life of more than 50 years, in contrast to an HDD's 5 to 10-year and magnetic tape's 15-20-year archival life. This allows data to be stored for longer periods of time without the need for data migration.</p>
  </blockquote>
  <p>More info on Blu-ray(tm) discs at
  <a href="https://panasonic.net/cns/blu-ray_disc/biz_ideal_media.html" rel="nofollow noreferrer">https://panasonic.net/cns/blu-ray_disc/biz_ideal_media.html</a></p>
  <p>Here are detailed info on Blu-ray(tm) and other disc tech by the Canadian governement, including comparisons and instructions for long term care:
  <a href="https://www.canada.ca/en/conservation-institute/services/conservation-preservation-publications/canadian-conservation-institute-notes/longevity-recordable-cds-dvds.html" rel="nofollow noreferrer">https://www.canada.ca/en/conservation-institute/services/conservation-preservation-publications/canadian-conservation-institute-notes/longevity-recordable-cds-dvds.html</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-21T18:17:25.520Z
Answers.LastActivityDate: 2022-10-15T16:26:19.167Z
Answers.LastEditDate: 2022-10-15T16:26:19.167Z
Answers.UserDisplayName: sm1
Answers.Body: <p>Aside from having one master hard drive as the backup of my field recording archive (on a portable hard drive), I've only just begun to archive my 15 years worth of field recordings to DVD. This is a laborous process however, and I am looking to Blu-ray as more efficient method. As other answers have noted, sole reliance on portable hard drives isn't a good idea. When a drive containing a comprehensive archive of recordings failed a few years ago, I narrowly avoided losing the lot.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-21T21:52:53.863Z
Answers.LastActivityDate: 2022-10-15T16:22:58.697Z
Answers.LastEditDate: 2022-10-15T16:22:58.697Z
Answers.UserDisplayName: Tristan Louth-Robins
Answers.Body: <p><a href="https://arbimon.rfcx.org/" rel="nofollow noreferrer">Arbimon</a> has pretty much unlimited free storage (though I believe the amount depends on whether it is public or private) and an uploader desktop app you can use to batch-upload in the background which is helpful. Also, Dataverse gives you up to 1TB of storage for free (figshare, Zenodo &amp; OSF are &lt;100GB for free).</p>
  <p>I'm also paranoid/old-school and back-up my data on external hard drives as well (see this post: <a href="https://bioacoustics.stackexchange.com/questions/790/reliable-hardrives-for-data-collection-and-storage/791#791">Reliable hardrives for data collection and storage</a>).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-22T13:27:43.710Z
Answers.LastActivityDate: 2022-07-22T13:27:43.710Z
Answers.UserDisplayName: Carly Batist
Answers.Body: <p>NOAA Alaska Fisheries Science Center acoustics also uses a Network Attached Storage (NAS) system. It is a good compromise between convenience of centralized data (HUGE advantage in sampling versus a bunch of hard drives in a cabinet) and fast enough I/O speed to still be able to read as if it's a hard disk for most purposes (to learn more, look up random reads). If you move to cloud based, you must abandon random reads or program in data staging to your processes which is something rarely thought about for most bioacousticians, has a high learning curve, and is a lot of work.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-27T18:20:09.170Z
Answers.LastActivityDate: 2022-07-27T18:20:09.170Z
Answers.UserDisplayName: Dan Woodrich
Answers.Body: <p><a href="https://www.ecosounds.org/" rel="nofollow noreferrer">Ecosounds</a> is a free archive for storing audio data.</p>
  <p>It currently has no limits and has just recently built a remote upload interface.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-28T09:33:06.773Z
Answers.LastActivityDate: 2022-07-28T09:33:06.773Z
Answers.UserDisplayName: Anthony Truskinger
Answers.Body: <p>Good question, and already quite a few useful answers! I think it is important to differentiate between storage and archiving. Unfortunately web storage media and hard drives seem not to be designed for long-term storage (decades or centuries!). Hard-drive storage facilitates quick retrieval and eventual change of data, but this is not needed for (digital) archives! As said in a former comment, petroglyphs are probably best, but Blu-ray seems to be the only available commercial product right now - we are still waiting for the holographic crystal storing Terabytes for ever and unchangeable in 1cm3!
  I think memory institutions (museums, and for bioacoustics eventually Natural History Museums) would be the best place to tackle this task, which is far beyond the capacity of individual researchers and/or universities.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-02T02:55:27.053Z
Answers.Text: That's a very relevant point re: difference between storage and archiving. I suppose this could also be framed as a question the user of the data could pose to themselves: is this data I will regularly be accessing (storage), or do I intend to simply file this away for an extended period (archiving.)
Answers.UserDisplayName: Tristan Louth-Robins
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-02T15:57:01.870Z
Answers.Text: Exactly! I have a lot of recordings which I would like to file away, on a pure storage  medium but available in a public archive/repository! Notabene: these data would be immune against hacking - data on CD cannot be changed, hence no ransom attacks possible. They also do not need energy during storage.
Answers.UserDisplayName: Klaus Riede
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-01T15:41:51.417Z
Answers.LastActivityDate: 2022-10-15T16:29:30.737Z
Answers.LastEditDate: 2022-10-15T16:29:30.737Z
Answers.UserDisplayName: Klaus Riede
Body: <p>I am curious how people are storing acoustic data in the long term. I know bioacousticians with piles of hard drives filled with decades of acoustic data lying around. But hard drives fail over time. <a href="https://www.backblaze.com/blog/how-long-do-disk-drives-last/" rel="noreferrer">One study</a> suggested that 90% of hard drives survive for three years and 80% for four years. I know of larger institutions that have server farms and climate-controlled rooms filled with drives (swanky!). But this isn't feasible for smaller organizations or individuals. How are you all maintaining your data over time?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-21T10:24:26.003Z
Comments.Text: Is this issue specific to acoustics :-) ?
Comments.UserDisplayName: Noil
Tags: <data-storage>
Title: Long-term storage of acoustic data
ThreadText: # Long-term storage of acoustic data
  
  (Asked by: username_0 on 2022-07-20)
  
  I am curious how people are storing acoustic data in the long term. I know bioacousticians with piles of hard drives filled with decades of acoustic data lying around. But hard drives fail over time. [One study](https://www.backblaze.com/blog/how-long-do-disk-drives-last/) suggested that 90% of hard drives survive for three years and 80% for four years. I know of larger institutions that have server farms and climate-controlled rooms filled with drives (swanky!). But this isn't feasible for smaller organizations or individuals. How are you all maintaining your data over time?
  
  **Tags:** data-storage
  
  > **Comments:**
  > **username_1**: Is this issue specific to acoustics :-) ?  
  
  ---
  
  ## 11 answers
  
  ### Answer by username_2 on 2022-07-20. Score: 20
  
  We have been using a Synology multibay NAS diskstation similar to this [one](https://www.synology.com/en-us/products/DS418). Additional expansion units such as [this](https://www.synology.com/en-us/products/DX517) can be added to daisy-chain to larger storage. Access to drives can be direct, via network, or you can also have an online access. Requires NAS drives (more expensive than normal internal hard drives). Over the years we have had several drives go bad-- our IT is alerted & they are able to change out the drives without any loss of data.
  
  In comparison, we have lost data in less than a year on other external hard drive storage devices (unlucky!).
  
  > **Comments:**
  > **username_3**: I, too, use the Synology NAS. I believe in redundancy, so I also set up cloud storage (through my institution) for my files.  But the NAS is where I put all my data first.  
  
  ---
  
  ### Answer by username_4 on 2022-07-20. Score: 15
  
  I have many hard drives and a backup hard drive for each of those, and for many of them, my collaborators have another copy. Hard drives have failed but the backups have saved the data.
  
  Another system like the Synology system is the Nexsan Beast which can hold up to 1 PB of data and has the option for expansion drives as well. It can also have up to two drives fail without any data loss, and those drives can be replaced as needed. It’s not inexpensive though and my understanding is it has a lifetime of 5-10 years.
  
  
  ---
  
  ### Answer by username_5 on 2022-07-21. Score: 11
  
  Someone said, the only reliable long-term storages of information are petroglyphs.
  
  I see three aspects:
  
  * media change over time and modern computers have even no proper interfaces anymore;
  * media size changes inviting recording of lots of noise (i.e. data without any information content);
  * media degrade over time (even petroglyphs).
  
  My suggestions would be:
  
  * when new more capable media emerge, spend some time in copying all relevant but only relevant old information to new media;
  * remove all data from your storage (declutter your storage) that is older than, say 10 years and nobody has looked into.
  
  AFAIK, continuous copying is the only way to fight physical degradation of media (not only tapes), and one can only copy so much data during a day.
  
  The 2nd suggestion sounds dramatic, but if there is no interest of your data then why keep it?
  
  OK, I know, my noise may be information for someone else, but that sounds very often only like an excuse to not to declutter your data store.
  
  > **Comments:**
  > **username_6**: Seconding the suggestion that getting rid of things that are unused is important! Assessing whether the issue is 'no one wants this' or 'no one can find this' is significant, but that is why collective archive projects are getting traction. There is only so much traffic that can result from 'contact the author' lines in publications  
  
  ---
  
  ### Answer by username_7 on 2022-07-21. Score: 7
  
  Thus far, I'm primarily using open public data archives as my way to preserve >5 years, since I know that most "local" solutions will fail or get lost. Openness means that *"lots of copies keeps stuff safe"*, as the saying goes. Good examples would be **Zenodo**, **OSF**, **Dataverse**, **Archive.org**.
  
  However, an issue with audio data (and other media) is size. Larger audio datasets can be >1TB and public services may be unwilling to provide for such sizes. I don't currently have a general answer to offer for that.
  
  > **Comments:**
  > **username_8**: If you are part of NOAA, the NODD is a resource that now supports data archival of large audio collections as active cloud storage on GCP. The caveat is that it must be up to publisher standards, publicly available, and relatively static. While it's still in it's fledgling stages it is available. If interested contact me or your science center IT reps.  
  
  ---
  
  ### Answer by username_9 on 2022-07-22. Score: 6
  
  Using NAS has also worked for us. It's relatively cheap (a 20 TB hardrive is ~500$), easy to set up, easy to use. You can set your NAS diskstation up in a way that if one or two of the drives fails your data is not lost.
  
  I would still be much happier if we had another longer term archiving option available to us. I have thought about **magnetic tape for data archiving**. Their lifespan should be ~ [30 years](https://www.techtarget.com/searchdatabackup/tip/How-to-estimate-the-lifespan-of-LTO-tapes) . The cost of the tapes themselves seems quite low, but the cost of the tape drives are a bit too much for us at the moment.
  
  
  ---
  
  ### Answer by username_10 on 2022-07-21. Score: 6
  
  For long-term storage, how about Blu-ray. Here's a sales pitch from Panasonic:
  
  > Panasonic Blu-ray™ discs have an archival life of more than 50 years, in contrast to an HDD's 5 to 10-year and magnetic tape's 15-20-year archival life. This allows data to be stored for longer periods of time without the need for data migration.
  
  More info on Blu-ray(tm) discs at
  <https://panasonic.net/cns/blu-ray_disc/biz_ideal_media.html>
  
  Here are detailed info on Blu-ray(tm) and other disc tech by the Canadian governement, including comparisons and instructions for long term care:
  <https://www.canada.ca/en/conservation-institute/services/conservation-preservation-publications/canadian-conservation-institute-notes/longevity-recordable-cds-dvds.html>
  
  
  ---
  
  ### Answer by username_11 on 2022-07-21. Score: 5
  
  Aside from having one master hard drive as the backup of my field recording archive (on a portable hard drive), I've only just begun to archive my 15 years worth of field recordings to DVD. This is a laborous process however, and I am looking to Blu-ray as more efficient method. As other answers have noted, sole reliance on portable hard drives isn't a good idea. When a drive containing a comprehensive archive of recordings failed a few years ago, I narrowly avoided losing the lot.
  
  
  ---
  
  ### Answer by username_12 on 2022-07-22. Score: 4
  
  [Arbimon](https://arbimon.rfcx.org/) has pretty much unlimited free storage (though I believe the amount depends on whether it is public or private) and an uploader desktop app you can use to batch-upload in the background which is helpful. Also, Dataverse gives you up to 1TB of storage for free (figshare, Zenodo & OSF are <100GB for free).
  
  I'm also paranoid/old-school and back-up my data on external hard drives as well (see this post: [Reliable hardrives for data collection and storage](https://bioacoustics.stackexchange.com/questions/790/reliable-hardrives-for-data-collection-and-storage/791#791)).
  
  
  ---
  
  ### Answer by username_8 on 2022-07-27. Score: 4
  
  NOAA Alaska Fisheries Science Center acoustics also uses a Network Attached Storage (NAS) system. It is a good compromise between convenience of centralized data (HUGE advantage in sampling versus a bunch of hard drives in a cabinet) and fast enough I/O speed to still be able to read as if it's a hard disk for most purposes (to learn more, look up random reads). If you move to cloud based, you must abandon random reads or program in data staging to your processes which is something rarely thought about for most bioacousticians, has a high learning curve, and is a lot of work.
  
  
  ---
  
  ### Answer by username_13 on 2022-07-28. Score: 3
  
  [Ecosounds](https://www.ecosounds.org/) is a free archive for storing audio data.
  
  It currently has no limits and has just recently built a remote upload interface.
  
  
  ---
  
  ### Answer by username_14 on 2022-08-01. Score: 3
  
  Good question, and already quite a few useful answers! I think it is important to differentiate between storage and archiving. Unfortunately web storage media and hard drives seem not to be designed for long-term storage (decades or centuries!). Hard-drive storage facilitates quick retrieval and eventual change of data, but this is not needed for (digital) archives! As said in a former comment, petroglyphs are probably best, but Blu-ray seems to be the only available commercial product right now - we are still waiting for the holographic crystal storing Terabytes for ever and unchangeable in 1cm3!
  I think memory institutions (museums, and for bioacoustics eventually Natural History Museums) would be the best place to tackle this task, which is far beyond the capacity of individual researchers and/or universities.
  
  > **Comments:**
  > **username_11**: That's a very relevant point re: difference between storage and archiving. I suppose this could also be framed as a question the user of the data could pose to themselves: is this data I will regularly be accessing (storage), or do I intend to simply file this away for an extended period (archiving.)  
  > **username_14**: Exactly! I have a lot of recordings which I would like to file away, on a pure storage  medium but available in a public archive/repository! Notabene: these data would be immune against hacking - data on CD cannot be changed, hence no ransom attacks possible. They also do not need energy during storage.  
  
  ---
  
  