Answers.Body: <p>WMXZ's answer covers the mathematics of the click detector very well, so I'll focus on the other part of the question which is about detection of sperm whale clicks specifically.</p>
  <p>In my experience, the default values for the click detector in Pamguard, as listed by WMXZ, work pretty well for sperm whale clicks. And while these parameters are specified in terms of samples, in my experience their efficacy for detecting sperm whale clicks seems to hold true across a range of sample rates -- which was somewhat surprising for me. I've used these default settings at sample rates from 8-96 kHz, and they've adequately detected sperm whale clicks in all cases -- at least for my applications.</p>
  <p>As WMXZ indicated, the mathematics of the detector are designed to adapt to varying background noise. Keeping the long filter small puts more emphasis on older samples. Keeping the short filter large puts more emphasis on the latest sample (i.e. the sharp rise from the onset of a click). And I agree that it's absolutely fine to experiment with the numbers to get a feel for how these parameters change things.</p>
  <p>But I also would like to caution against wasting time with unnecessary optimization -- particularly the part of the question about reducing false positives. For some reason, it's common for people to get spend an unwarranted amount of time trying optimize their detector and wring out every last bit of performance, when in reality all that is actually required <em>for their application</em> is to characterise the performance of the detector.</p>
  <p>So this raises some additional questions such as: what is the overall goal? what should be considered a false positive? and how many false positives can be tolerated whilst still achieving this goal?</p>
  <p>For example, is the ultimate goal to make a statement about ecology? or signal processing? Is it important to detect every single sperm whale click? Or is it acceptable just to know whether sperm whales were detected during that hour of the day? or day of the month? or month of the year? The approach to each of these things can differ greatly. So maybe tread with caution to avoid spending a lot of time trying to optimise a parameter that ultimately doesn't make a difference in terms of answering your question.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-22T23:22:30.033Z
Answers.LastActivityDate: 2024-12-22T23:22:30.033Z
Answers.UserDisplayName: Brian Miller
Answers.Body: <p>The actual Parameter settings are</p>
  <ul>
  <li>long filter: 1e-5</li>
  <li>long filter2: 1e-6</li>
  <li>short filter: 1e-1</li>
  </ul>
  <p>As indicated in the Pamguard Help, the filters are simple exponential moving averages (EMA)</p>
  <p><span class="math-container">$$ x_f = x_f + \alpha (|x|-x_f)$$</span></p>
  <p>where <span class="math-container">$\alpha$</span> is the number given above. A small value supresses the contribution of the measurements.</p>
  <p>My own setting for filters like these would be that the long_filter_2 would always be 1/10th of the noise alone (long) filter. The long and short filter should really depend on the sampling frequency (for noise) and the signal duration for the short filter.</p>
  <p>A reasonable formula for <span class="math-container">$\alpha$</span> could be (in analogy to financial world)
  <span class="math-container">$$\alpha = 2/(1+nsamp)$$</span></p>
  <p>where <span class="math-container">$nsamp$</span> is the number of samples you wanted to average. For noise this could for example be equivalent to 0.1s (nsamp=0.1*fsamp) with fsamp being the sampling frequency and for signal I would not average more than 10 samples.</p>
  <p>comment 1: I would then use 0.5 as parameter for short filter.</p>
  <p>comment 2: There may be other formulas for estimating <span class="math-container">$\alpha$</span>, but the idea is the same, even if the numbers change. In the end, it is worthwhile to experiment with different numbers.</p>
  <p>comment 3: The detector is adaptive as it adapts to varying background noise.</p>
  <p>comment 4: One can argue if the filtering should not be done in the power domain, where noise averaging makes more (physical) sense, but the Pamguard developper have decided otherwise.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-22T18:50:50.837Z
Answers.LastActivityDate: 2024-12-22T21:13:51.533Z
Answers.LastEditDate: 2024-12-22T21:13:51.533Z
Answers.UserDisplayName: WMXZ
Body: <p>I am configuring the <strong>Click Detector</strong> module in PAMGuard to detect sperm whale clicks in a noisy environment. The PAMGuard manual mentions that:</p>
  <ul>
  <li>The Long Filter calculates the noise level when no clicks are active.</li>
  <li>The Long Filter 2 is used to calculate the noise level during click detection.</li>
  <li>The Short Filter measures the signal level in a short time window.</li>
  </ul>
  <p>Click detection occurs when the S/N ratio exceeds the Threshold value.</p>
  <p>I have some questions:</p>
  <ol>
  <li>How do I choose appropriate values for the Long Filter and Long Filter 2 in noisy environments?</li>
  <li>How can I adjust these filters to reduce false positives in an environment with variable noise (e.g., from vessels)?</li>
  </ol>
  <p>I would appreciate understanding how these parameters interact with each other and what adjustments could improve detection for sperm whale clicks specifically.</p>
  <p>Thanks for your help!</p>
  <hr />
  <p>This should lead to helpful, detailed answers that clarify the interplay between these parameters in PAMGuard. </p>
  
Tags: <pamguard>
Title: How do the Long Filter, Long Filter 2, and Short Filter parameters affect detection in the PAMGuard Click Detector module?
ThreadText: # How do the Long Filter, Long Filter 2, and Short Filter parameters affect detection in the PAMGuard Click Detector module?
  
  (Asked by: username_0 on 2024-12-22)
  
  I am configuring the **Click Detector** module in PAMGuard to detect sperm whale clicks in a noisy environment. The PAMGuard manual mentions that:
  
  * The Long Filter calculates the noise level when no clicks are active.
  * The Long Filter 2 is used to calculate the noise level during click detection.
  * The Short Filter measures the signal level in a short time window.
  
  Click detection occurs when the S/N ratio exceeds the Threshold value.
  
  I have some questions:
  
  1. How do I choose appropriate values for the Long Filter and Long Filter 2 in noisy environments?
  2. How can I adjust these filters to reduce false positives in an environment with variable noise (e.g., from vessels)?
  
  I would appreciate understanding how these parameters interact with each other and what adjustments could improve detection for sperm whale clicks specifically.
  
  Thanks for your help!
  
  ---
  
  This should lead to helpful, detailed answers that clarify the interplay between these parameters in PAMGuard.
  
  **Tags:** pamguard
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-12-22. Score: 1
  
  username_2's answer covers the mathematics of the click detector very well, so I'll focus on the other part of the question which is about detection of sperm whale clicks specifically.
  
  In my experience, the default values for the click detector in Pamguard, as listed by username_2, work pretty well for sperm whale clicks. And while these parameters are specified in terms of samples, in my experience their efficacy for detecting sperm whale clicks seems to hold true across a range of sample rates -- which was somewhat surprising for me. I've used these default settings at sample rates from 8-96 kHz, and they've adequately detected sperm whale clicks in all cases -- at least for my applications.
  
  As username_2 indicated, the mathematics of the detector are designed to adapt to varying background noise. Keeping the long filter small puts more emphasis on older samples. Keeping the short filter large puts more emphasis on the latest sample (i.e. the sharp rise from the onset of a click). And I agree that it's absolutely fine to experiment with the numbers to get a feel for how these parameters change things.
  
  But I also would like to caution against wasting time with unnecessary optimization -- particularly the part of the question about reducing false positives. For some reason, it's common for people to get spend an unwarranted amount of time trying optimize their detector and wring out every last bit of performance, when in reality all that is actually required *for their application* is to characterise the performance of the detector.
  
  So this raises some additional questions such as: what is the overall goal? what should be considered a false positive? and how many false positives can be tolerated whilst still achieving this goal?
  
  For example, is the ultimate goal to make a statement about ecology? or signal processing? Is it important to detect every single sperm whale click? Or is it acceptable just to know whether sperm whales were detected during that hour of the day? or day of the month? or month of the year? The approach to each of these things can differ greatly. So maybe tread with caution to avoid spending a lot of time trying to optimise a parameter that ultimately doesn't make a difference in terms of answering your question.
  
  
  ---
  
  ### Answer by username_2 on 2024-12-22. Score: 1
  
  The actual Parameter settings are
  
  * long filter: 1e-5
  * long filter2: 1e-6
  * short filter: 1e-1
  
  As indicated in the Pamguard Help, the filters are simple exponential moving averages (EMA)
  
  $$ x\_f = x\_f + \alpha (|x|-x\_f)$$
  
  where $\alpha$ is the number given above. A small value supresses the contribution of the measurements.
  
  My own setting for filters like these would be that the long\_filter\_2 would always be 1/10th of the noise alone (long) filter. The long and short filter should really depend on the sampling frequency (for noise) and the signal duration for the short filter.
  
  A reasonable formula for $\alpha$ could be (in analogy to financial world)
  $$\alpha = 2/(1+nsamp)$$
  
  where $nsamp$ is the number of samples you wanted to average. For noise this could for example be equivalent to 0.1s (nsamp=0.1\*fsamp) with fsamp being the sampling frequency and for signal I would not average more than 10 samples.
  
  comment 1: I would then use 0.5 as parameter for short filter.
  
  comment 2: There may be other formulas for estimating $\alpha$, but the idea is the same, even if the numbers change. In the end, it is worthwhile to experiment with different numbers.
  
  comment 3: The detector is adaptive as it adapts to varying background noise.
  
  comment 4: One can argue if the filtering should not be done in the power domain, where noise averaging makes more (physical) sense, but the Pamguard developper have decided otherwise.
  
  
  ---
  
  