Answers.Body: <p>The SAMBAH project in the Baltic processed 400 years of C-POD data and the detection performance - the effective detection radius - was estimated using more than one method to give a population estimate of porpoises.
  Data was collected over parts of 2 years from 300 sites and 60% of these sites, in a very coherent spatial pattern, had zero detections despite logging porpiose like clicks (because the clicks were not in trains).</p>
  <p>The KERNO-F classifier for the F-POD does have sensitivity controls that the user can tweak (not recommended) but, as the author of it, I would not use them to contruct ROC curves because ROC curves are specific to noise regimes as well as instruments, and sampling a sufficient spread of noise regimes to produce a generalised value would be a vast project giving a result that might not fit any specific location!</p>
  <p>The classifier is long and I would not dream of assessing it by unpacking it.  It's only possible to develop and optimise such algorithms empirically, and that is essentially what neural networks and most machine leraning does.
  In this case the classifier has been emprically tuned to work in a wide range of adverse environments, so that most users will get data that, after validation of a sample of detections, can be used without editing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-09T07:39:16.943Z
Answers.LastActivityDate: 2022-07-09T07:39:16.943Z
Answers.UserDisplayName: NickT
Answers.Body: <p>Not to my knowledge. I have explicitly asked the creator of this software while at a conference workshop about this, and they answered me like a politician. I think that they have zero interest in producing an ROC curve while their sales are hot. Would be keen to hear if there have been efforts to do so, perhaps in the grey literature.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-01T12:56:22.427Z
Answers.LastActivityDate: 2022-07-01T12:56:22.427Z
Answers.UserDisplayName: Chloe
Body: <p>Outside of <a href="https://www.tandfonline.com/doi/full/10.1080/09524622.2018.1477071" rel="noreferrer">Carsten et al., 2019</a>, are there any good papers that detail the performance and detection probability of the KERNO classifier included in CPOD.exe? I'm aware of multiple papers that compare the classifier to broad-band recordings and open-access classifiers (below), and that certainly gives us insight. However, I have never seen a ROC curve provided by the KERNO authors in a peer-reviewed study.</p>
  <p><strong>References</strong></p>
  <p>Clausen K.T., Tougaard J., Carstensen J., Delefosse M., Teilmann J. (2019) Noise affects porpoise click detections – the magnitude of the effect depends on logger type and detection filter settings. Bioacoustics 28(5): 443-458.</p>
  <p>Sarnocinska, J., Tougaard, J., Johnson, M., Madsen, P. T., &amp; Wahlberg, M. (2016, July). Comparing the performance of C-PODs and SoundTrap/PAMGUARD in detecting the acoustic activity of harbor porpoises (Phocoena phocoena). In Proceedings of Meetings on Acoustics 4ENAL (Vol. 27, No. 1, p. 070013). Acoustical Society of America.</p>
  <p>Jacobson, E. K., Merkens, K. P., Forney, K. A., &amp; Barlow, J. (2017). Comparison of harbor porpoise (Phocoena phocoena) echolocation clicks recorded simultaneously on two passive acoustic monitoring instruments. NOAA-TM-NMFS-SWFSC-583; <a href="http://doi.org/10.7289/V5/TM-SWFSC-583" rel="noreferrer">http://doi.org/10.7289/V5/TM-SWFSC-583</a></p>
  
Tags: <automated-detection><cpod><classifier>
Title: Performance review and detection probability of the CPOD KERNO classifier?
ThreadText: # Performance review and detection probability of the CPOD KERNO classifier?
  
  (Asked by: username_0 on 2022-06-27)
  
  Outside of [Carsten et al., 2019](https://www.tandfonline.com/doi/full/10.1080/09524622.2018.1477071), are there any good papers that detail the performance and detection probability of the KERNO classifier included in CPOD.exe? I'm aware of multiple papers that compare the classifier to broad-band recordings and open-access classifiers (below), and that certainly gives us insight. However, I have never seen a ROC curve provided by the KERNO authors in a peer-reviewed study.
  
  **References**
  
  Clausen K.T., Tougaard J., Carstensen J., Delefosse M., Teilmann J. (2019) Noise affects porpoise click detections – the magnitude of the effect depends on logger type and detection filter settings. Bioacoustics 28(5): 443-458.
  
  Sarnocinska, J., Tougaard, J., Johnson, M., Madsen, P. T., & Wahlberg, M. (2016, July). Comparing the performance of C-PODs and SoundTrap/PAMGUARD in detecting the acoustic activity of harbor porpoises (Phocoena phocoena). In Proceedings of Meetings on Acoustics 4ENAL (Vol. 27, No. 1, p. 070013). Acoustical Society of America.
  
  Jacobson, E. K., Merkens, K. P., Forney, K. A., & Barlow, J. (2017). Comparison of harbor porpoise (Phocoena phocoena) echolocation clicks recorded simultaneously on two passive acoustic monitoring instruments. NOAA-TM-NMFS-SWFSC-583; <http://doi.org/10.7289/V5/TM-SWFSC-583>
  
  **Tags:** automated-detection, cpod, classifier
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-07-09. Score: 3
  
  The SAMBAH project in the Baltic processed 400 years of C-POD data and the detection performance - the effective detection radius - was estimated using more than one method to give a population estimate of porpoises.
  Data was collected over parts of 2 years from 300 sites and 60% of these sites, in a very coherent spatial pattern, had zero detections despite logging porpiose like clicks (because the clicks were not in trains).
  
  The KERNO-F classifier for the F-POD does have sensitivity controls that the user can tweak (not recommended) but, as the author of it, I would not use them to contruct ROC curves because ROC curves are specific to noise regimes as well as instruments, and sampling a sufficient spread of noise regimes to produce a generalised value would be a vast project giving a result that might not fit any specific location!
  
  The classifier is long and I would not dream of assessing it by unpacking it. It's only possible to develop and optimise such algorithms empirically, and that is essentially what neural networks and most machine leraning does.
  In this case the classifier has been emprically tuned to work in a wide range of adverse environments, so that most users will get data that, after validation of a sample of detections, can be used without editing.
  
  
  ---
  
  ### Answer by username_2 on 2022-07-01. Score: 2
  
  Not to my knowledge. I have explicitly asked the creator of this software while at a conference workshop about this, and they answered me like a politician. I think that they have zero interest in producing an ROC curve while their sales are hot. Would be keen to hear if there have been efforts to do so, perhaps in the grey literature.
  
  
  ---
  
  