Answers.Body: <p>The method I would try to use is is based on the cumulative energy method. It assumes that there is some noise only data before and after the signal of interest.</p>
  <p>I had, by chance one of the sea-bird chick data, from which I extracted a short sequence of a very weak call, so I tried the following R-script (note I'm not an R-user, so it may be improved)</p>
  <pre><code>library('tuneR')
  library('signal')
  
  filename&lt;-'C:/Users/xxxxx/Desktop/F2023_07_31_072000_chickclip_2.wav'
  #
  data &lt;- readWave(filename)
  xx &lt;- dat@left
  N &lt;- length(xx)
  tt &lt;- seq(0,N-1,1)/fs
  
  plot(tt,xx,type='l', xlab = 'Time (s)', ylab = ' ')
  
  # determine sample rate
  fs = data@samp.rate
  
  # number of points to use for the fft
  nfft=1024
  
  # window size (in points)
  window=256
  
  # overlap (in points)
  overlap=128
  
  # create spectrogram
  spec1 = specgram(x = xx,
                   n = nfft,
                   Fs = fs,
                   window = window,
                   overlap = overlap)
  # discard phase information
  P = abs(spec1$S)
  
  # normalize
  P = P/max(P)
  
  # convert to dB
  P = 10*log10(P)
  
  # config time axis
  t = spec1<span class="math-container">$t
  # plot spectrogram
  image(x = t,
        y = spec1$</span>f/1000,
        z = t(P),
        ylab = 'Frequency [kHz]',
        xlab = 'Time [s]',
  )
  
  #--------------------------------------------
  xx &lt;- xx[2:N]-xx[1:N-1]
  N &lt;- length(xx)
  tt &lt;- seq(0,N-1,1)/fs
  
  plot(tt,xx,type='l', xlab = 'Time (s)', ylab = ' ')
  
  #-----------------------------
  bf &lt;- butter(8, c(2,10)/24, type=&quot;pass&quot;)
  yy &lt;- filtfilt(bf, xx)
  
  zz &lt;- yy
  
  N &lt;- length(zz)
  tt &lt;- seq(0,N-1,1)/fs
  #
  uu &lt;- cumsum(zz^2)
  vv &lt;- uu/uu[N] 
  
  isig &lt;- which(vv&gt;=0.01 &amp; vv&lt;0.99)
  
  i1=isig[1]
  i2=isig[length(isig)]
  
  plot(tt,vv,type='l', xlab = 'Time (s)', ylab = ' ')
  points(tt[i1],vv[i1],col='red',bg='red')
  points(tt[i2],vv[i2],col='red',bg='red')
  
  
  #---------------------------------------------
                   
  # create spectrogram
  spec2 = specgram(x = zz,
                  n = nfft,
                  Fs = fs,
                  window = window,
                  overlap = overlap)
  
  # discard phase information
  P = abs(spec$S)
  
  # normalize
  P = P/max(P)
  
  # convert to dB
  P = 10*log10(P)
  
  # config time axis
  t = spec$t
  
  # plot spectrogram
  image(x = t,
        y = spec$f/1000,
        z = t(P),
        ylab = 'Frequency [kHz]',
        xlab = 'Time [s]',
  )
  lines(c(tt[i1],tt[i1]), c(2,10))
  lines(c(tt[i2],tt[i2]), c(2,10))
  </code></pre>
  <p>which generated the following plots
  <a href="https://i.sstatic.net/82eug9DT.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/82eug9DT.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/J7cxX32C.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/J7cxX32C.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/M6U4YdOp.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/M6U4YdOp.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/26PwtVsM.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/26PwtVsM.png" alt="enter image description here" /></a>
  <a href="https://i.sstatic.net/pBJzKarf.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/pBJzKarf.png" alt="enter image description here" /></a></p>
  <p>The first plot is the original data, followed by a spectrogram.
  The third plot shows the filtered data made to remove out of band data and equalize the background noise.
  The fourth plot shows the cumulative energy with 1 and 99 percentiles and the last plot gives marks the start and stop times on the filtered spectrogram.</p>
  <p>After, determine start and stop times, the resulting snippet can then be saved as wav file, or processed further.</p>
  <p><strong>Edit</strong>: Selecting the signal bandwidth and start and stop percentile requires some experimentation (that is why I show all the spectrograms). Both percentiles depend on the signal-to-noise ratio (SNR) and should for low SNR be choosen to narrow the signal (i.e. 5 and 95%). While the starting percentile (here 1%) is in general a little bit easier to define, the stop percentile can be tricky, especially if reverberation increases the signal reception. Reverberation is easily noted when noise trailing the signal is higher than the noise before the signal. As @BrianMiller noted, visual inspection is always good.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-10T00:20:31.730Z
Answers.Text: Great answer & exactly what I meant by "energy-based approach." When applying automated methods like this, it's important to verify the results to confirm the solution to one issue doesn't introduce new issues. My dataset had a variety of noise types and a fairly wide range of SNR. In practice this meant there wasn't a fixed pair of percentiles that worked for all data. One could take this a step further and try to estimate/quantify that relationship  to pick the appropriate percentile for a given SNR. Allternatively, one could just apply a fixed percentile with full manual review (what I did)
Answers.UserDisplayName: Brian Miller
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-10T06:18:57.293Z
Answers.Text: @Brian, I realized that you had "limited-to-moderate success" with this method, but I thought that it would be worthwhile to present it. I added a comment on percentile selection.
Answers.UserDisplayName: WMXZ
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-10T06:38:37.240Z
Answers.Text: Agree it's worthwhile to show the answer. I didn't bother to do so because I suspected from your comment you were planning a response, and knew that you'd do a better job of it than me :-)
Answers.UserDisplayName: Brian Miller
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-09T15:47:47.043Z
Answers.LastActivityDate: 2024-05-10T06:31:09.890Z
Answers.LastEditDate: 2024-05-10T06:31:09.890Z
Answers.UserDisplayName: WMXZ
Answers.Body: <p>I've had the same issue in needing tight time-frequency bounds for estimation of source level of blue whale sounds.</p>
  <p>My dataset had around 2000 calls, and I spent an hour exploring a few automated (energy-based) approaches with limited-to-moderate success.</p>
  <p>My dataset didn't have great SNR, and did have a lot of reverberation, so the start and stop of many calls were difficult to pick. I wasn't aware of <code>tailor_sels</code> from warbleR until reading your post. But looking at the documentation, it seems to match the approach that I ultimately took.</p>
  <p>In the end, I concluded it would be quicker and more reliable just to redraw the bounds manually, and this also gave me another opportunity to filter out calls that contained other sources of noise. Ultimately, it took about two hours of focused effort using a custom Matlab script of my own making to relabel the calls (essentially just a loop with audioread, spectrogram, &amp; ginput if the psuedocode &amp; Matlab function names have any value here).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-08T18:45:05.447Z
Answers.Text: Hi Brian, thanks for your comment. That helps me give a good idea for the processing time of adjusting these selections! I've previously used the Acoustat program in Matlab for vocalisations with really high background noise (common murres foraging at sea), and only had success once I filtered out the constant low-frequency noise, so I understand that particular struggle as well.
Answers.UserDisplayName: Katrina W
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-08T01:29:42.243Z
Answers.LastActivityDate: 2024-05-08T01:29:42.243Z
Answers.UserDisplayName: Brian Miller
Body: <p>I have drawn &gt;4000 selections on seabird chick call spectrograms in Raven with ~0.1 to 0.4 seconds on either side of the signal (a stipulation of the call parameter measurement software I was previously going to use). I have now switched to using the warbleR package in R to measure call parameters, which requires that the imported Raven selections have no empty space on either side of the selection. I can manually tailor every selection using the tailor_sels function in warbleR but would much prefer to automate the process. I have tried using the auto_detec function, but many of the function's abilities have been deprecated and transferred to ohun. Auto_detec also seems to have difficulties with the negative DC offset from my microphone (constant noise under 0.5kHz).</p>
  <p>I have looked into ohun as well, which aims to automatically detect sound events (either energy or template-based detection). However, this requires a training dataset of selections with proper start/end times (no space on either side), and I am not trying to 'detect' calls across all recordings as I've already selected all the calls of interest. There doesn't seem to be a function in ohun that I could use the auto_detec function to only run the detection within my selections.</p>
  <p>Has anyone ever had this issue before and know how to manipulate functions in either ohun or warbleR to automate this process? Please let me know if I could add any spectrograms or example code for clarity.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-07T18:39:51.177Z
Comments.Text: Katrina, are you willing to preprocess the data in R to fit the snippet to only contain the signal?
Comments.UserDisplayName: WMXZ
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-08T18:39:52.393Z
Comments.Text: Hi Walter, yes I am willing to preprocess the selections and shorten them to only contain the signal (if that is what you are asking?). It seems like this might be the fastest way forward as well.
Comments.UserDisplayName: Katrina W
Tags: <r><automated-detection><methods><signal-detection><ohun>
Title: How do I shorten call selections to only contain the signal and not background noise using warbleR or ohun in R?
ThreadText: # How do I shorten call selections to only contain the signal and not background noise using warbleR or ohun in R?
  
  (Asked by: username_0 on 2024-05-07)
  
  I have drawn >4000 selections on seabird chick call spectrograms in Raven with ~0.1 to 0.4 seconds on either side of the signal (a stipulation of the call parameter measurement software I was previously going to use). I have now switched to using the warbleR package in R to measure call parameters, which requires that the imported Raven selections have no empty space on either side of the selection. I can manually tailor every selection using the tailor\_sels function in warbleR but would much prefer to automate the process. I have tried using the auto\_detec function, but many of the function's abilities have been deprecated and transferred to ohun. Auto\_detec also seems to have difficulties with the negative DC offset from my microphone (constant noise under 0.5kHz).
  
  I have looked into ohun as well, which aims to automatically detect sound events (either energy or template-based detection). However, this requires a training dataset of selections with proper start/end times (no space on either side), and I am not trying to 'detect' calls across all recordings as I've already selected all the calls of interest. There doesn't seem to be a function in ohun that I could use the auto\_detec function to only run the detection within my selections.
  
  Has anyone ever had this issue before and know how to manipulate functions in either ohun or warbleR to automate this process? Please let me know if I could add any spectrograms or example code for clarity.
  
  **Tags:** r, automated-detection, methods, signal-detection, ohun
  
  > **Comments:**
  > **username_1**: Katrina, are you willing to preprocess the data in R to fit the snippet to only contain the signal?  
  > **username_0**: Hi Walter, yes I am willing to preprocess the selections and shorten them to only contain the signal (if that is what you are asking?). It seems like this might be the fastest way forward as well.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-05-09. Score: 2
  
  The method I would try to use is is based on the cumulative energy method. It assumes that there is some noise only data before and after the signal of interest.
  I had, by chance one of the sea-bird chick data, from which I extracted a short sequence of a very weak call, so I tried the following R-script (note I'm not an R-user, so it may be improved)
  library('tuneR')
  library('signal')
  
  filename<-'C:/Users/xxxxx/Desktop/F2023_07_31_072000_chickclip_2.wav'
  #
  data <- readWave(filename)
  xx <- dat@left
  N <- length(xx)
  tt <- seq(0,N-1,1)/fs
  
  plot(tt,xx,type='l', xlab = 'Time (s)', ylab = ' ')
  
  # determine sample rate
  fs = data@samp.rate
  
  # number of points to use for the fft
  nfft=1024
  
  # window size (in points)
  window=256
  
  # overlap (in points)
  overlap=128
  
  # create spectrogram
  spec1 = specgram(x = xx,
                   n = nfft,
                   Fs = fs,
                   window = window,
                   overlap = overlap)
  # discard phase information
  P = abs(spec1$S)
  
  # normalize
  P = P/max(P)
  
  # convert to dB
  P = 10*log10(P)
  
  # config time axis
  t = spec1$t
  # plot spectrogram
  image(x = t,
        y = spec1$f/1000,
        z = t(P),
        ylab = 'Frequency [kHz]',
        xlab = 'Time [s]',
  )
  
  #--------------------------------------------
  xx <- xx[2:N]-xx[1:N-1]
  N <- length(xx)
  tt <- seq(0,N-1,1)/fs
  
  plot(tt,xx,type='l', xlab = 'Time (s)', ylab = ' ')
  
  #-----------------------------
  bf <- butter(8, c(2,10)/24, type="pass")
  yy <- filtfilt(bf, xx)
  
  zz <- yy
  
  N <- length(zz)
  tt <- seq(0,N-1,1)/fs
  #
  uu <- cumsum(zz^2)
  vv <- uu/uu[N] 
  
  isig <- which(vv>=0.01 & vv<0.99)
  
  i1=isig[1]
  i2=isig[length(isig)]
  
  plot(tt,vv,type='l', xlab = 'Time (s)', ylab = ' ')
  points(tt[i1],vv[i1],col='red',bg='red')
  points(tt[i2],vv[i2],col='red',bg='red')
  
  
  #---------------------------------------------
                   
  # create spectrogram
  spec2 = specgram(x = zz,
                  n = nfft,
                  Fs = fs,
                  window = window,
                  overlap = overlap)
  
  # discard phase information
  P = abs(spec$S)
  
  # normalize
  P = P/max(P)
  
  # convert to dB
  P = 10*log10(P)
  
  # config time axis
  t = spec$t
  
  # plot spectrogram
  image(x = t,
        y = spec$f/1000,
        z = t(P),
        ylab = 'Frequency [kHz]',
        xlab = 'Time [s]',
  )
  lines(c(tt[i1],tt[i1]), c(2,10))
  lines(c(tt[i2],tt[i2]), c(2,10))
  
  which generated the following plots
  
  
  
  
  
  The first plot is the original data, followed by a spectrogram.
  The third plot shows the filtered data made to remove out of band data and equalize the background noise.
  The fourth plot shows the cumulative energy with 1 and 99 percentiles and the last plot gives marks the start and stop times on the filtered spectrogram.
  After, determine start and stop times, the resulting snippet can then be saved as wav file, or processed further.
  Edit: Selecting the signal bandwidth and start and stop percentile requires some experimentation (that is why I show all the spectrograms). Both percentiles depend on the signal-to-noise ratio (SNR) and should for low SNR be choosen to narrow the signal (i.e. 5 and 95%). While the starting percentile (here 1%) is in general a little bit easier to define, the stop percentile can be tricky, especially if reverberation increases the signal reception. Reverberation is easily noted when noise trailing the signal is higher than the noise before the signal. As @BrianMiller noted, visual inspection is always good.
  
  > **Comments:**
  > **username_2**: Great answer & exactly what I meant by "energy-based approach." When applying automated methods like this, it's important to verify the results to confirm the solution to one issue doesn't introduce new issues. My dataset had a variety of noise types and a fairly wide range of SNR. In practice this meant there wasn't a fixed pair of percentiles that worked for all data. One could take this a step further and try to estimate/quantify that relationship  to pick the appropriate percentile for a given SNR. Allternatively, one could just apply a fixed percentile with full manual review (what I did)  
  > **username_1**: @Brian, I realized that you had "limited-to-moderate success" with this method, but I thought that it would be worthwhile to present it. I added a comment on percentile selection.  
  > **username_2**: Agree it's worthwhile to show the answer. I didn't bother to do so because I suspected from your comment you were planning a response, and knew that you'd do a better job of it than me :-)  
  
  ---
  
  ### Answer by username_2 on 2024-05-08. Score: 2
  
  I've had the same issue in needing tight time-frequency bounds for estimation of source level of blue whale sounds.
  
  My dataset had around 2000 calls, and I spent an hour exploring a few automated (energy-based) approaches with limited-to-moderate success.
  
  My dataset didn't have great SNR, and did have a lot of reverberation, so the start and stop of many calls were difficult to pick. I wasn't aware of `tailor_sels` from warbleR until reading your post. But looking at the documentation, it seems to match the approach that I ultimately took.
  
  In the end, I concluded it would be quicker and more reliable just to redraw the bounds manually, and this also gave me another opportunity to filter out calls that contained other sources of noise. Ultimately, it took about two hours of focused effort using a custom Matlab script of my own making to relabel the calls (essentially just a loop with audioread, spectrogram, & ginput if the psuedocode & Matlab function names have any value here).
  
  > **Comments:**
  > **username_0**: Hi Brian, thanks for your comment. That helps me give a good idea for the processing time of adjusting these selections! I've previously used the Acoustat program in Matlab for vocalisations with really high background noise (common murres foraging at sea), and only had success once I filtered out the constant low-frequency noise, so I understand that particular struggle as well.  
  
  ---
  
  