Answers.Body: <p>Perhaps have a look at Ollie Metcalf's <code>hardRain</code> R package. It's designed to identify recordings with rain, I think based on passing an amplitude threshold between certain frequency bands. I'm sure you could modify it to pick out wind recordings instead. I don't know if it would be possible to discriminate wind from snow scooters, but it does have the benefit of being quite straightforward.</p>
  <p>Metcalf et al (2020) &quot;hardRain: An R package for quick, automated rainfall detection in ecoacoustic datasets using a threshold-based approach&quot;, Ecological Indicators. <a href="https://www.sciencedirect.com/science/article/pii/S1470160X19307873?casa_token=L13wgMv0i_cAAAAA:jn0YHr7l8v0VeSTrNSuU-FYe5hW6IGrc89qWPZ8BxyiZglxs64rcLptpC4KdQSzctSKsQ_B7dw" rel="nofollow noreferrer">https://www.sciencedirect.com/science/article/pii/S1470160X19307873?casa_token=L13wgMv0i_cAAAAA:jn0YHr7l8v0VeSTrNSuU-FYe5hW6IGrc89qWPZ8BxyiZglxs64rcLptpC4KdQSzctSKsQ_B7dw</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-28T07:31:14.810Z
Answers.LastActivityDate: 2022-09-28T07:31:14.810Z
Answers.UserDisplayName: EcologyTom
Answers.Body: <p>In line with the thinking of looking at the frequency content, you could take that a little further: I assume that wind noise generally has more energy at higher frequencies than snowscooters, thus the spectra should be shifted a bit to higher frequencies (but both probably broadband in nature). If you use a reasonably small FFT bin, your spectrum will be smooth-ish (low frequency resolution). Alternatively you could run an envelope function over a spectrum (e.g. Hilbert transform).
  Now you can run a correlation test between your recorded spectrum (probe/test in below figure) and some typical wind or scooter envelopes/spectra-shapes. In the below we have identified the noise as a scooter.
  <a href="https://i.sstatic.net/GrFuj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/GrFuj.png" alt="enter image description here" /></a>
  I'm inclined to think that scooters have some strong tonals in their noise, given the use of rotary components. If true this opens up using kurtosis to have a measure of the &quot;peakedness&quot; of the spectrum (or maybe simply a running mean threshold will do).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-28T07:57:52.680Z
Answers.LastActivityDate: 2022-09-28T14:01:49.273Z
Answers.LastEditDate: 2022-09-28T14:01:49.273Z
Answers.UserDisplayName: Rasmus
Answers.Body: <p>I have in my other answer advocated for improving the deep learning model directly. But in case one wishes to use a separate model to filter out wind-noise, here is one approach:</p>
  <p>Use a generic audio classifier model to detect wind noise. For example, the AudioSet ontology includes the class <a href="http://research.google.com/audioset/ontology/wind_1.html" rel="nofollow noreferrer">Wind</a>, and also the subclass <a href="http://research.google.com/audioset/ontology/wind_noise_microphone_1.html" rel="nofollow noreferrer">Wind noise (Microphone)</a>. There are several pre-trained models available that do AudioSet classification, with pretty good performance overall and are easy to use. Examples include <a href="https://github.com/qiuqiangkong/audioset_tagging_cnn/tree/master" rel="nofollow noreferrer">PANNs</a> and <a href="https://tfhub.dev/google/yamnet/1" rel="nofollow noreferrer">YAMNet</a>.</p>
  <p>If the performance of those models are not satisfactory out-of-the-box, it is possible to fine-tune them on your data. Use the models to extract a vector embedding from the audio, and then use a small dataset with wind-noise and not. Some 100-1000 labeled samples might be enough. For that approach, one can also use vector embedding mores such as <a href="https://github.com/marl/openl3" rel="nofollow noreferrer">OpenL3</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-18T21:49:00.720Z
Answers.Text: There is now a question about this kind of filtering specifically: https://bioacoustics.stackexchange.com/questions/1434/detection-and-removing-of-windy-events-in-wild-acoustic-recordings/
Answers.UserDisplayName: Jon Nordby
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-04T19:02:29.953Z
Answers.LastActivityDate: 2023-06-04T19:02:29.953Z
Answers.UserDisplayName: Jon Nordby
Answers.Body: <p>Adding more wind samples to the &quot;non-scooter&quot; / ambient noise class is probably the best solution (your first suggestion). That will make your existing model strictly better at these cases, without needing other pre- or post-processing steps.</p>
  <p>But, it might be difficult to get enough samples to really make the network /
  training process take those into account.
  If that is the case, there are some tricks that you can do:</p>
  <ul>
  <li>Use <em>sample weighting</em> during training to make these samples get higher priority. In Tensorflow/Keras this can be done by passing the <code>sample_weight</code> argument to the <a href="https://keras.io/api/models/model_training_apis/#fit-method" rel="nofollow noreferrer">fit() method</a>.</li>
  <li>Use data-augmentation as a data balancing strategy (upsampling). Creating multiple samples with slight variations for the problematic/challenging cases.</li>
  <li>Use audio-mixing of event sounds (such as the wind noise) onto background samples, at different SNR to create more challenging background samples. Can be done using tools like <a href="https://github.com/justinsalamon/scaper" rel="nofollow noreferrer">Scaper</a>.</li>
  <li>If you have a lot of simple/repetitive samples represented in the dataset, consider dropping some of them, to allow more focus on the tricky cases. This is a data balancing strategy based on undersampling.</li>
  </ul>
  <p>Note that the particular failure-mode that you have identified (wind noise being used for), might be a symptom of a wider issue. It might be that your model is just be looking at (any kind of) soundlevel variation, or spectrogram variation - and decide that means &quot;snow-scooter&quot;.</p>
  <p>To diagnose and prevent that, consider including also other &quot;event like&quot; sound into (besides the wind noise) into your dataset. Preferably, things that could plausibly occur in your deployment scenario.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-04T18:31:16.503Z
Answers.LastActivityDate: 2023-06-08T08:53:35.643Z
Answers.LastEditDate: 2023-06-08T08:53:35.643Z
Answers.UserDisplayName: Jon Nordby
Body: <p>I am currently working on building a deep learning model for snowscooter detections. The model does fairly well at detecting snowscooters but <strong>it triggers on wind noises</strong>, increasing the rate of false detections. Do you have any experience with this kind of problems and how do you deal with it?</p>
  <p>The solutions I was thinking of include:</p>
  <ul>
  <li>Adding more wind noises when training the model so it can properly discriminate between snowscooter noises and wind noises.</li>
  <li>Using a high pass filter for filtering out the wind noises first and then use the model on the audio file</li>
  <li>For each file, compute a &quot;wind noise&quot; (is there any entropy metric for such purpose?) metric and if it is too windy simply don't analyse the specific file</li>
  <li>Use a model for detecting the segments containing wind and delete these segments. Is there any open source model for this?</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-28T18:54:14.063Z
Comments.Text: What are the spectral profiles of your snowscooters vs wind? The methods that might work if they overlap a lot are very different than if they don't. Adding spectrograms of each to your question might be helpful here.
Comments.UserDisplayName: dtsavage
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-09T11:28:33.140Z
Comments.Text: Adding spectrograms is important especially if they give also timescales.  Also any type of pre-processing, signal conditioning should be mentioned. My hunch is that the applied procedure is using constant threshold and ignores background noise levels.
Comments.UserDisplayName: WMXZ
Tags: <machine-learning><automated-detection><classifier>
Title: Filtering out wind from audio recordings
ThreadText: # Filtering out wind from audio recordings
  
  (Asked by: username_0 on 2022-09-28)
  
  I am currently working on building a deep learning model for snowscooter detections. The model does fairly well at detecting snowscooters but **it triggers on wind noises**, increasing the rate of false detections. Do you have any experience with this kind of problems and how do you deal with it?
  
  The solutions I was thinking of include:
  
  * Adding more wind noises when training the model so it can properly discriminate between snowscooter noises and wind noises.
  * Using a high pass filter for filtering out the wind noises first and then use the model on the audio file
  * For each file, compute a "wind noise" (is there any entropy metric for such purpose?) metric and if it is too windy simply don't analyse the specific file
  * Use a model for detecting the segments containing wind and delete these segments. Is there any open source model for this?
  
  **Tags:** machine-learning, automated-detection, classifier
  
  > **Comments:**
  > **username_1**: What are the spectral profiles of your snowscooters vs wind? The methods that might work if they overlap a lot are very different than if they don't. Adding spectrograms of each to your question might be helpful here.  
  > **username_2**: Adding spectrograms is important especially if they give also timescales.  Also any type of pre-processing, signal conditioning should be mentioned. My hunch is that the applied procedure is using constant threshold and ignores background noise levels.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_3 on 2022-09-28. Score: 4
  
  Perhaps have a look at Ollie Metcalf's `hardRain` R package. It's designed to identify recordings with rain, I think based on passing an amplitude threshold between certain frequency bands. I'm sure you could modify it to pick out wind recordings instead. I don't know if it would be possible to discriminate wind from snow scooters, but it does have the benefit of being quite straightforward.
  
  Metcalf et al (2020) "hardRain: An R package for quick, automated rainfall detection in ecoacoustic datasets using a threshold-based approach", Ecological Indicators. <https://www.sciencedirect.com/science/article/pii/S1470160X19307873?casa_token=L13wgMv0i_cAAAAA:jn0YHr7l8v0VeSTrNSuU-FYe5hW6IGrc89qWPZ8BxyiZglxs64rcLptpC4KdQSzctSKsQ_B7dw>
  
  
  ---
  
  ### Answer by username_4 on 2022-09-28. Score: 2
  
  In line with the thinking of looking at the frequency content, you could take that a little further: I assume that wind noise generally has more energy at higher frequencies than snowscooters, thus the spectra should be shifted a bit to higher frequencies (but both probably broadband in nature). If you use a reasonably small FFT bin, your spectrum will be smooth-ish (low frequency resolution). Alternatively you could run an envelope function over a spectrum (e.g. Hilbert transform).
  Now you can run a correlation test between your recorded spectrum (probe/test in below figure) and some typical wind or scooter envelopes/spectra-shapes. In the below we have identified the noise as a scooter.
  [![enter image description here](https://i.sstatic.net/GrFuj.png)](https://i.sstatic.net/GrFuj.png)
  I'm inclined to think that scooters have some strong tonals in their noise, given the use of rotary components. If true this opens up using kurtosis to have a measure of the "peakedness" of the spectrum (or maybe simply a running mean threshold will do).
  
  
  ---
  
  ### Answer by username_5 on 2023-06-04. Score: 1
  
  I have in my other answer advocated for improving the deep learning model directly. But in case one wishes to use a separate model to filter out wind-noise, here is one approach:
  
  Use a generic audio classifier model to detect wind noise. For example, the AudioSet ontology includes the class [Wind](http://research.google.com/audioset/ontology/wind_1.html), and also the subclass [Wind noise (Microphone)](http://research.google.com/audioset/ontology/wind_noise_microphone_1.html). There are several pre-trained models available that do AudioSet classification, with pretty good performance overall and are easy to use. Examples include [PANNs](https://github.com/qiuqiangkong/audioset_tagging_cnn/tree/master) and [YAMNet](https://tfhub.dev/google/yamnet/1).
  
  If the performance of those models are not satisfactory out-of-the-box, it is possible to fine-tune them on your data. Use the models to extract a vector embedding from the audio, and then use a small dataset with wind-noise and not. Some 100-1000 labeled samples might be enough. For that approach, one can also use vector embedding mores such as [OpenL3](https://github.com/marl/openl3).
  
  > **Comments:**
  > **username_5**: There is now a question about this kind of filtering specifically: https://bioacoustics.stackexchange.com/questions/1434/detection-and-removing-of-windy-events-in-wild-acoustic-recordings/  
  
  ---
  
  ### Answer by username_5 on 2023-06-04. Score: 1
  
  Adding more wind samples to the "non-scooter" / ambient noise class is probably the best solution (your first suggestion). That will make your existing model strictly better at these cases, without needing other pre- or post-processing steps.
  
  But, it might be difficult to get enough samples to really make the network /
  training process take those into account.
  If that is the case, there are some tricks that you can do:
  
  * Use *sample weighting* during training to make these samples get higher priority. In Tensorflow/Keras this can be done by passing the `sample_weight` argument to the [fit() method](https://keras.io/api/models/model_training_apis/#fit-method).
  * Use data-augmentation as a data balancing strategy (upsampling). Creating multiple samples with slight variations for the problematic/challenging cases.
  * Use audio-mixing of event sounds (such as the wind noise) onto background samples, at different SNR to create more challenging background samples. Can be done using tools like [Scaper](https://github.com/justinsalamon/scaper).
  * If you have a lot of simple/repetitive samples represented in the dataset, consider dropping some of them, to allow more focus on the tricky cases. This is a data balancing strategy based on undersampling.
  
  Note that the particular failure-mode that you have identified (wind noise being used for), might be a symptom of a wider issue. It might be that your model is just be looking at (any kind of) soundlevel variation, or spectrogram variation - and decide that means "snow-scooter".
  
  To diagnose and prevent that, consider including also other "event like" sound into (besides the wind noise) into your dataset. Preferably, things that could plausibly occur in your deployment scenario.
  
  
  ---
  
  