Answers.Body: <p>I find that 5 min file sizes sampled at 500 kHz (2-3 channels) do well in terms of collecting data in the field. In terms of processing, I find at the moment it has more to do with the computer's amount of RAM, how much you've preallocated for PAMGuard, and where your soundfiles are stored. For best processing, have your files stored on an external hard drive attached to your local machine.</p>
  <p>You can change the amount of RAM allocated to PAMGuard by opening up PAMGuard Viewer's .ini file and modifying the first two lines of code. I tend to set up my computer on the conservative side so I set my max RAM to one half of my computer’s total RAM. This allows me the ability to do other tasks with my computer if I need. To set the maximum of RAM to allot to Pamguard, change the value on line 2. To change how much RAM is allocated in general to Pamguard, change the value on line 1.</p>
  <p>Example .ini file:</p>
  <pre><code>-Xms2048m
  -Xmx8192m
  -Djava.io.tmpdir=&quot;%USERPROFILE%\Pamguard&quot;
  -Djava.library.path=&quot;C:\pamguard\PamguardBeta_2_02_06\lib64&quot;
  -jar &quot;C:\pamguard\PamguardBeta_2_02_06\Pamguard-2.02.06.jar&quot;
  -smru
  </code></pre>
  <p><em>*note lines 4 and 5 are specific to where PAMGuard is installed on your computer so that might be a little different</em></p>
  <p><a href="https://i.sstatic.net/E3ImG.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/E3ImG.png" alt="The highlighted files can all be changed to increase the RAM with the 'Xmx' setting" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-05T02:33:50.913Z
Answers.LastActivityDate: 2023-01-13T02:21:10.870Z
Answers.LastEditDate: 2023-01-13T02:21:10.870Z
Answers.UserDisplayName: A. DeAngelis
Answers.Body: <p>At high data rates (&gt;192 kHz), I usually stick with shorter files (1-2 minutes). PamGuard has no problem stitching these together as if one continuous file. You'll find that these files are easier to share with colleagues than the maximum file size. Also, some software will give you problems with very large files.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-13T17:44:58.463Z
Answers.LastActivityDate: 2022-07-13T17:44:58.463Z
Answers.UserDisplayName: Jay
Answers.Body: <p>I'd generally stick to 2 GBytes, sometime less, depending on plans for further processing the wav files.</p>
  <p>There is a fundamental limit to wav file file sizes of 4 Gigabytes. This is because the size of the data is stored as an unsigned 32 bit integer in the wav file header and this number simply can't get any bigger than 2^32 bytes. However, some languages (Java being an example) don't support unsigned integers, so are effectively limited to sizes of 2^31 which is about 2 Gigabytes (PAMGuard has a modified wav stream reader since the standard Java one fails with files &gt; 2Gbytes).</p>
  <p>If you plan further analysis with Matlab, when you read the files into Matlab, the default audioread(...) function converts the 16 bit integer values in the wav files to 64 bit double precision values, which use four times as much memory. Therefore reading in a whole 2Gbyte wav file into Matlab may require 8Gbytes of RAM (16 for a 4Gbyte wav file). If you've lots of RAM, this may not matter too much and there are ways around this, such as only reading part of the file. However, if you're planning to use software which will read a whole file at a time into memory, I'd recommend a much smaller file size.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-10T13:04:34.533Z
Answers.LastActivityDate: 2022-07-10T13:04:34.533Z
Answers.UserDisplayName: Doug Gillespie
Body: <p>I would like to maximize the file size of my wav files for continuous recordings without significantly sacrificing the performance of PAMGuard analysis.</p>
  <p>I realize this depends computer specs (among other things) and that there is no one right answer.</p>
  <p>Are there any specific PAMGuard limitations with regards to file sizes, or points where you find processing starts to suffer?</p>
  
Tags: <pamguard><analysis><wav>
Title: Maximum wav file size for reasonable processing in PAMGuard
ThreadText: # Maximum wav file size for reasonable processing in PAMGuard
  
  (Asked by: username_0 on 2022-07-04)
  
  I would like to maximize the file size of my wav files for continuous recordings without significantly sacrificing the performance of PAMGuard analysis.
  
  I realize this depends computer specs (among other things) and that there is no one right answer.
  
  Are there any specific PAMGuard limitations with regards to file sizes, or points where you find processing starts to suffer?
  
  **Tags:** pamguard, analysis, wav
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2022-07-05. Score: 14
  
  I find that 5 min file sizes sampled at 500 kHz (2-3 channels) do well in terms of collecting data in the field. In terms of processing, I find at the moment it has more to do with the computer's amount of RAM, how much you've preallocated for PAMGuard, and where your soundfiles are stored. For best processing, have your files stored on an external hard drive attached to your local machine.
  You can change the amount of RAM allocated to PAMGuard by opening up PAMGuard Viewer's .ini file and modifying the first two lines of code. I tend to set up my computer on the conservative side so I set my max RAM to one half of my computer’s total RAM. This allows me the ability to do other tasks with my computer if I need. To set the maximum of RAM to allot to Pamguard, change the value on line 2. To change how much RAM is allocated in general to Pamguard, change the value on line 1.
  Example .ini file:
  -Xms2048m
  -Xmx8192m
  -Djava.io.tmpdir="%USERPROFILE%\Pamguard"
  -Djava.library.path="C:\pamguard\PamguardBeta_2_02_06\lib64"
  -jar "C:\pamguard\PamguardBeta_2_02_06\Pamguard-2.02.06.jar"
  -smru
  
  *note lines 4 and 5 are specific to where PAMGuard is installed on your computer so that might be a little different
  
  
  ---
  
  ### Answer by username_2 on 2022-07-13. Score: 5
  
  At high data rates (>192 kHz), I usually stick with shorter files (1-2 minutes). PamGuard has no problem stitching these together as if one continuous file. You'll find that these files are easier to share with colleagues than the maximum file size. Also, some software will give you problems with very large files.
  
  
  ---
  
  ### Answer by username_3 on 2022-07-10. Score: 5
  
  I'd generally stick to 2 GBytes, sometime less, depending on plans for further processing the wav files.
  
  There is a fundamental limit to wav file file sizes of 4 Gigabytes. This is because the size of the data is stored as an unsigned 32 bit integer in the wav file header and this number simply can't get any bigger than 2^32 bytes. However, some languages (Java being an example) don't support unsigned integers, so are effectively limited to sizes of 2^31 which is about 2 Gigabytes (PAMGuard has a modified wav stream reader since the standard Java one fails with files > 2Gbytes).
  
  If you plan further analysis with Matlab, when you read the files into Matlab, the default audioread(...) function converts the 16 bit integer values in the wav files to 64 bit double precision values, which use four times as much memory. Therefore reading in a whole 2Gbyte wav file into Matlab may require 8Gbytes of RAM (16 for a 4Gbyte wav file). If you've lots of RAM, this may not matter too much and there are ways around this, such as only reading part of the file. However, if you're planning to use software which will read a whole file at a time into memory, I'd recommend a much smaller file size.
  
  
  ---
  
  