Body: <p>I currently have made a convolutional network module in torch that makes the basis functions in the usual way <code>fourier_basis = np.fft.fft(np.eye(frame_size))</code> and I use this in the forward &quot;FFT&quot; to encode an audio signal into the frequency domain. This is a component in an autoencoder, so there is a corresponding inverse transform using inverse 1d convs.</p>
  <p>However, I suspect that the set of periods in the normal FFT basis functions are not ideal for my domain, and I'd like to test that by creating an autoencoder that has learnable basis functions (perfect sinusoids) that differ in frequency.</p>
  <p>It seems fairly simple to create a &quot;free&quot; filterbank that has just room for a bunch of parameters and the network figures out the values, <a href="https://github.com/asteroid-team/asteroid-filterbanks/blob/master/asteroid_filterbanks/free_fb.py#L28" rel="nofollow noreferrer">like in the asteroid filterbanks API</a>.</p>
  <p>But in my case I want the basis functions to be sine waves, not totally freeform waveforms, and let the amplitude and frequencies of the sines be differentiable.</p>
  <p>Strangely, I haven't found any implementations online, which leads me to believe either this is a) a bad idea, or b) I am not looking in the right places or describing this concept correctly.</p>
  <p><strong>the question:</strong> how can I implement a differentiable convolutional FFT module where the sinusoidal basis functions have learnable frequency, phase, and amplitude? (I'm looking for some insight on how to implement, not a full code example necessarily.)</p>
  <p>thanks!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-08T11:27:10.620Z
Comments.Text: What is your question? Please, edit the post to ask a specific question and also put it in the title. Thanks
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-08T19:42:59.350Z
Comments.Text: ok I added the question and edited the title. thanks!
Comments.UserDisplayName: lollercoaster
Tags: <convolutional-neural-networks><pytorch><filters><audio-processing><signal-processing>
Title: How to implement differentiable sinusoidal basis functions for a convolutional FFT module in torch
UserDisplayName: lollercoaster
ThreadText: # How to implement differentiable sinusoidal basis functions for a convolutional FFT module in torch
  
  (Asked by: username_0 on 2024-08-08)
  
  I currently have made a convolutional network module in torch that makes the basis functions in the usual way `fourier_basis = np.fft.fft(np.eye(frame_size))` and I use this in the forward "FFT" to encode an audio signal into the frequency domain. This is a component in an autoencoder, so there is a corresponding inverse transform using inverse 1d convs.
  
  However, I suspect that the set of periods in the normal FFT basis functions are not ideal for my domain, and I'd like to test that by creating an autoencoder that has learnable basis functions (perfect sinusoids) that differ in frequency.
  
  It seems fairly simple to create a "free" filterbank that has just room for a bunch of parameters and the network figures out the values, [like in the asteroid filterbanks API](https://github.com/asteroid-team/asteroid-filterbanks/blob/master/asteroid_filterbanks/free_fb.py#L28).
  
  But in my case I want the basis functions to be sine waves, not totally freeform waveforms, and let the amplitude and frequencies of the sines be differentiable.
  
  Strangely, I haven't found any implementations online, which leads me to believe either this is a) a bad idea, or b) I am not looking in the right places or describing this concept correctly.
  
  **the question:** how can I implement a differentiable convolutional FFT module where the sinusoidal basis functions have learnable frequency, phase, and amplitude? (I'm looking for some insight on how to implement, not a full code example necessarily.)
  
  thanks!
  
  **Tags:** convolutional-neural-networks, pytorch, filters, audio-processing, signal-processing
  > **Comments:**
  > **username_1**: What is your question? Please, edit the post to ask a specific question and also put it in the title. Thanks  
  > **username_0**: ok I added the question and edited the title. thanks!  
  
  ---
  
  ## 0 answers
  
  