Answers.Body: <p>Since you say that you &quot;believe that a neural network architecture is needed here...&quot;, I am assuming that you are open to other options.  This approach doesn't utilize a neural networks, but I think it can potentially get the job done (with some caveats).  One approach is to</p>
  <ol>
  <li>First isolate the unwanted sound segments that you want to &quot;subtract&quot;</li>
  <li>Calculate the Fourier transform of each unwanted sound segment</li>
  <li>Determine the components with the highest amplitudes in each unwanted sound segment</li>
  <li>Determine if these dominant peaks have shared frequencies across the various unwanted sound segments</li>
  <li>If so, this spectrum of frequencies is the &quot;signature&quot; of your unwanted sound</li>
  <li>Once you have that, you can simply subtract those frequencies from your full sound recording to remove the unwanted sound</li>
  <li>This is done by finding the Fourier transform of the full sound recording</li>
  <li>Subtracting the frequencies (i.e., the signature) of the unwanted sound</li>
  <li>Transforming back to the time domain to get the signal minus the unwanted sound</li>
  </ol>
  <p>As implied, for this approach to work, the unwanted sounds will have to have a consistent set of dominant frequencies (i.e., a signature).  If you have multiple &quot;types&quot; of unwanted sounds, then this procedure can be repeated for each type (i.e., find the signature for each type).  If you have multiple types of unwanted sounds and you cannot distinguish them, a neural network may then be useful.</p>
  <p>You will also have to make sure that those frequencies are not a significant component of your desired signal.  Otherwise, you will be removing an important part of your desired signal.  To do that, just follow the same procedure as described for the unwanted signal.</p>
  <p>This code from <a href="https://www.mathworks.com/matlabcentral/answers/382113-how-to-remove-noise-from-audio-using-fourier-transform-and-filter-and-to-obtain-back-the-original-au" rel="nofollow noreferrer">MATLAB</a> describes one possible implementation (albeit for a denoising application).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-29T04:10:52.477Z
Answers.LastActivityDate: 2022-11-29T04:40:23.293Z
Answers.LastEditDate: 2022-11-29T04:40:23.293Z
Answers.UserDisplayName: Snehal Patel
Body: <p>I have a use case where I will be inputting audio to a model, and the output of the model will be the same audio except with certain sounds removed (volume set to zero).  The dataset is generated by taking an audio file, duplicating it, and then zeroing out the unwanted sounds (usually a half second long).</p>
  <p>I believe a neural network architecture is needed here with the input being the undisturbed audio and its spectrogram.  The output is then the modified/cleaned audio.</p>
  <p>What model architectures would work for this use case?  I would potentially like to have this run real-time as a person is speaking.</p>
  
Tags: <convolutional-neural-networks><transformer><time-series><audio-processing>
Title: What type of neural network architecture allows filtering out of unwanted sounds?
UserDisplayName: HonestMath
ThreadText: # What type of neural network architecture allows filtering out of unwanted sounds?
  
  (Asked by: username_0 on 2022-11-29)
  
  I have a use case where I will be inputting audio to a model, and the output of the model will be the same audio except with certain sounds removed (volume set to zero). The dataset is generated by taking an audio file, duplicating it, and then zeroing out the unwanted sounds (usually a half second long).
  
  I believe a neural network architecture is needed here with the input being the undisturbed audio and its spectrogram. The output is then the modified/cleaned audio.
  
  What model architectures would work for this use case? I would potentially like to have this run real-time as a person is speaking.
  
  **Tags:** convolutional-neural-networks, transformer, time-series, audio-processing
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-11-29. Score: 1
  
  Since you say that you "believe that a neural network architecture is needed here...", I am assuming that you are open to other options. This approach doesn't utilize a neural networks, but I think it can potentially get the job done (with some caveats). One approach is to
  
  1. First isolate the unwanted sound segments that you want to "subtract"
  2. Calculate the Fourier transform of each unwanted sound segment
  3. Determine the components with the highest amplitudes in each unwanted sound segment
  4. Determine if these dominant peaks have shared frequencies across the various unwanted sound segments
  5. If so, this spectrum of frequencies is the "signature" of your unwanted sound
  6. Once you have that, you can simply subtract those frequencies from your full sound recording to remove the unwanted sound
  7. This is done by finding the Fourier transform of the full sound recording
  8. Subtracting the frequencies (i.e., the signature) of the unwanted sound
  9. Transforming back to the time domain to get the signal minus the unwanted sound
  
  As implied, for this approach to work, the unwanted sounds will have to have a consistent set of dominant frequencies (i.e., a signature). If you have multiple "types" of unwanted sounds, then this procedure can be repeated for each type (i.e., find the signature for each type). If you have multiple types of unwanted sounds and you cannot distinguish them, a neural network may then be useful.
  
  You will also have to make sure that those frequencies are not a significant component of your desired signal. Otherwise, you will be removing an important part of your desired signal. To do that, just follow the same procedure as described for the unwanted signal.
  
  This code from [MATLAB](https://www.mathworks.com/matlabcentral/answers/382113-how-to-remove-noise-from-audio-using-fourier-transform-and-filter-and-to-obtain-back-the-original-au) describes one possible implementation (albeit for a denoising application).
  
  
  ---
  
  