Answers.Body: <p>I have worked with relatively large FLAC data sets (a few TB, not anywhere near 500 TB!) and been very happy with it/have no issues to report. I am not an engineer so am coming at this opinion from the analyst end user side of things. I find FLAC is very easy to work with and don't notice a slow down in processing time, with the caveat that I haven't done any timed tests, and that I am primarily working in MATLAB.</p>
  <p>The <a href="http://embeddedocean.com/passive-acoustics-2/" rel="noreferrer">WISPR system</a> that I've worked with extensively over the last several years records directly to FLAC. In our first years with that project we were converting the FLAC to wav for any bioacoustic analysis. But, in recent years we stopped doing that because we found it unnecessary; most programs can now read FLAC directly (MATLAB, Raven, Audition) and I modified a version of Triton that can make LTSAs directly from FLAC files (I have it forked from the main Triton repository on <a href="https://github.com/sfregosi/Triton/tree/flac" rel="noreferrer">GitHub</a> and am always looking for people to try it and point out bugs :)).</p>
  <p>I have not personally compressed wavs to FLAC using the executable mentioned in this <a href="https://bioacoustics.stackexchange.com/a/350/42">answer</a>, but I have gone from wav to FLAC and back just using <code>audioread</code> and <code>audiowrite</code> in MATLAB. I assume that is not the fastest choice, but I find it is fast enough for a few TB of data. I</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-08T21:18:04.200Z
Answers.LastActivityDate: 2022-07-08T21:18:04.200Z
Answers.UserDisplayName: selene
Answers.Body: <p>Yes, I almost always store large acoustic files in FLAC. It's usually not an issue for smaller files of 10s or even 100s of GB, but files in the TB+ range get unwieldy fast.  And moving them is like watching paint dry.</p>
  <p>If you have data in the 100s of TB, you will definitely notice your data transfer time significantly decrease...and of course your storage costs. As mentioned above, most acoustic software lets you work directly from FLAC.  If it doesn't, it will provide a conversion function. On modern systems, I haven't noticed that much of a performance hit.  Conversion is pretty easy to script in just about any modern language.</p>
  <p>Besides the decreased storage requirements, FLAC is a lossless compression, it's open source, and it's available for all major platforms directly from the people that maintain it (<a href="https://xiph.org/flac/" rel="noreferrer">https://xiph.org/flac/</a>)...or your favorite software. I personally like the idea that it isn't proprietary.</p>
  <p>If you are planning to submit acoustic data to NOAA/NCEI, they prefer FLAC.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-09T02:42:11.203Z
Answers.LastActivityDate: 2022-07-09T02:42:11.203Z
Answers.UserDisplayName: CosmicSpittle
Answers.Body: <p>FLAC is great. I hope to see more projects and application programs using it. Compressing takes a bit of CPU time but uncompressing is easy. Depending on noise level, I get a compression to about 1/2 or 1/3.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-08T06:38:28.767Z
Answers.LastActivityDate: 2022-07-08T06:38:28.767Z
Answers.UserDisplayName: Jörg Rychen
Answers.Body: <p>Our usual workflow with FLAC is to leave projects that are not being actively analyzed saved in FLAC, which gives us about a 50% file size reduction, while projects that are being actively analyzed stay in WAV format because it's easier to work with, especially in R with seewave and tuneR.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-08T23:16:33.237Z
Answers.LastActivityDate: 2022-07-08T23:16:33.237Z
Answers.UserDisplayName: dtsavage
Body: <p>We have a large (&gt;500TB) acoustic dataset that so far has remained uncompressed. I am considering going back and FLAC compressing all of our data in order to make data transfers and archive easier and more cost-effective. However, it is active storage and the data are still regularly accessed and used for different projects. MATLAB will read FLAC data using <em>audioread.m</em> and I think PAMGuard will read FLAC files as well. I don't know a ton about data compression, will you lose data processing efficiency if you directly read in FLAC files without uncompressing to WAV? Either because it takes longer to read from FLAC, or because the file is being uncompressed and held in memory? Are there other factors that make FLAC compression not feasible for actively used data storage?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-08T22:36:15.333Z
Comments.Text: A couple previous posts on different forms of compression, file types, etc. - https://bioacoustics.stackexchange.com/questions/344/how-if-at-all-are-you-compressing-your-recordings AND https://bioacoustics.stackexchange.com/questions/48/does-the-file-format-of-audio-recordings-e-g-wav-vs-mp3-matter-if-you-are-p/62#62
Comments.UserDisplayName: Carly Batist
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-20T22:18:42.687Z
Comments.Text: @AnnAllen -- If one of the answers below answers your question, please 'accept' the answer by checking the checkmark under the vote tally. See https://stackoverflow.com/help/someone-answers
Comments.UserDisplayName: Shannon
Tags: <recording><archive><data-storage><compression>
Title: Does anyone regularly use FLAC compression for their acoustic data?
ThreadText: # Does anyone regularly use FLAC compression for their acoustic data?
  
  (Asked by: username_0 on 2022-07-08)
  
  We have a large (>500TB) acoustic dataset that so far has remained uncompressed. I am considering going back and FLAC compressing all of our data in order to make data transfers and archive easier and more cost-effective. However, it is active storage and the data are still regularly accessed and used for different projects. MATLAB will read FLAC data using *audioread.m* and I think PAMGuard will read FLAC files as well. I don't know a ton about data compression, will you lose data processing efficiency if you directly read in FLAC files without uncompressing to WAV? Either because it takes longer to read from FLAC, or because the file is being uncompressed and held in memory? Are there other factors that make FLAC compression not feasible for actively used data storage?
  
  **Tags:** recording, archive, data-storage, compression
  
  > **Comments:**
  > **username_1**: A couple previous posts on different forms of compression, file types, etc. - https://bioacoustics.stackexchange.com/questions/344/how-if-at-all-are-you-compressing-your-recordings AND https://bioacoustics.stackexchange.com/questions/48/does-the-file-format-of-audio-recordings-e-g-wav-vs-mp3-matter-if-you-are-p/62#62  
  > **username_2**: @username_0 -- If one of the answers below answers your question, please 'accept' the answer by checking the checkmark under the vote tally. See https://stackoverflow.com/help/someone-answers  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_3 on 2022-07-08. Score: 8
  
  I have worked with relatively large FLAC data sets (a few TB, not anywhere near 500 TB!) and been very happy with it/have no issues to report. I am not an engineer so am coming at this opinion from the analyst end user side of things. I find FLAC is very easy to work with and don't notice a slow down in processing time, with the caveat that I haven't done any timed tests, and that I am primarily working in MATLAB.
  
  The [WISPR system](http://embeddedocean.com/passive-acoustics-2/) that I've worked with extensively over the last several years records directly to FLAC. In our first years with that project we were converting the FLAC to wav for any bioacoustic analysis. But, in recent years we stopped doing that because we found it unnecessary; most programs can now read FLAC directly (MATLAB, Raven, Audition) and I modified a version of Triton that can make LTSAs directly from FLAC files (I have it forked from the main Triton repository on [GitHub](https://github.com/sfregosi/Triton/tree/flac) and am always looking for people to try it and point out bugs :)).
  
  I have not personally compressed wavs to FLAC using the executable mentioned in this [answer](https://bioacoustics.stackexchange.com/a/350/42), but I have gone from wav to FLAC and back just using `audioread` and `audiowrite` in MATLAB. I assume that is not the fastest choice, but I find it is fast enough for a few TB of data. I
  
  
  ---
  
  ### Answer by username_4 on 2022-07-09. Score: 5
  
  Yes, I almost always store large acoustic files in FLAC. It's usually not an issue for smaller files of 10s or even 100s of GB, but files in the TB+ range get unwieldy fast. And moving them is like watching paint dry.
  
  If you have data in the 100s of TB, you will definitely notice your data transfer time significantly decrease...and of course your storage costs. As mentioned above, most acoustic software lets you work directly from FLAC. If it doesn't, it will provide a conversion function. On modern systems, I haven't noticed that much of a performance hit. Conversion is pretty easy to script in just about any modern language.
  
  Besides the decreased storage requirements, FLAC is a lossless compression, it's open source, and it's available for all major platforms directly from the people that maintain it (<https://xiph.org/flac/>)...or your favorite software. I personally like the idea that it isn't proprietary.
  
  If you are planning to submit acoustic data to NOAA/NCEI, they prefer FLAC.
  
  
  ---
  
  ### Answer by username_5 on 2022-07-08. Score: 4
  
  FLAC is great. I hope to see more projects and application programs using it. Compressing takes a bit of CPU time but uncompressing is easy. Depending on noise level, I get a compression to about 1/2 or 1/3.
  
  
  ---
  
  ### Answer by username_6 on 2022-07-08. Score: 3
  
  Our usual workflow with FLAC is to leave projects that are not being actively analyzed saved in FLAC, which gives us about a 50% file size reduction, while projects that are being actively analyzed stay in WAV format because it's easier to work with, especially in R with seewave and tuneR.
  
  
  ---
  
  