Answers.Body: <p>Note that using complex numbers doubles the number of network parameters. In general, we can say that a network with <span class="math-container">$n$</span> complex nodes will have a training cost equivalent to a network with <span class="math-container">$2n$</span> real nodes.</p>
  <p>About audio, it is true that a lot of analytics is done in Fourier/Laplace spaces, where complex numbers are mandatory. However, the incoming signal is real, also filters can be keep on this field. It is not usual, by example, the need of a DFT followed by a complex number NN.</p>
  <p>Finally, note one of the main advantages of a NN is their non-linear structure and capabilities. Usual transforms and related tools are restricted to linear. In this point, the NNs are a step after.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-20T10:28:09.227Z
Answers.LastActivityDate: 2022-07-23T09:37:53.903Z
Answers.LastEditDate: 2022-07-23T09:37:53.903Z
Answers.UserDisplayName: pasaba por aqui
Body: <p>If you switch a neural network from real weights to complex weights, you're roughly doubling the size of the network, and increasing the computational load by a factor of 2 to 4. My question is, in general, roughly how does the benefit of using complex weights stack up to those extra costs? E.g. Will a complex neural network with half the weights achieve worse/comparable/better performance than a regular network with real weights?</p>
  <p>In audio signal processing, complex numbers make the theory much more elegant, which is why I imagine using complex numbers might be disproportionately beneficial. Though I can also imagine the complexity they introduce might overly hinder things as well.</p>
  <p>As far as I know, no one uses complex weights in the NNs (which must be for a reason), but I'd like a more definitive answer.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-20T06:22:39.080Z
Comments.Text: related: https://ai.stackexchange.com/questions/7247/why-do-we-need-floats-for-using-neural-networks. Also see this paper: https://arxiv.org/abs/1705.09792 And it looks like someone has asked the same question already in dataStackExchange: https://datascience.stackexchange.com/questions/28676/are-there-neural-networks-packages-that-use-complex-numbers
Comments.UserDisplayName: SpiderRico
Tags: <neural-networks><machine-learning><audio-processing><linear-algebra><signal-processing>
Title: How does using complex weights in a neural network affect performance?
UserDisplayName: chausies
ThreadText: # How does using complex weights in a neural network affect performance?
  
  (Asked by: username_0 on 2022-02-20)
  
  If you switch a neural network from real weights to complex weights, you're roughly doubling the size of the network, and increasing the computational load by a factor of 2 to 4. My question is, in general, roughly how does the benefit of using complex weights stack up to those extra costs? E.g. Will a complex neural network with half the weights achieve worse/comparable/better performance than a regular network with real weights?
  
  In audio signal processing, complex numbers make the theory much more elegant, which is why I imagine using complex numbers might be disproportionately beneficial. Though I can also imagine the complexity they introduce might overly hinder things as well.
  
  As far as I know, no one uses complex weights in the NNs (which must be for a reason), but I'd like a more definitive answer.
  
  **Tags:** neural-networks, machine-learning, audio-processing, linear-algebra, signal-processing
  > **Comments:**
  > **username_1**: related: https://ai.stackexchange.com/questions/7247/why-do-we-need-floats-for-using-neural-networks. Also see this paper: https://arxiv.org/abs/1705.09792 And it looks like someone has asked the same question already in dataStackExchange: https://datascience.stackexchange.com/questions/28676/are-there-neural-networks-packages-that-use-complex-numbers  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-02-20. Score: 0
  
  Note that using complex numbers doubles the number of network parameters. In general, we can say that a network with $n$ complex nodes will have a training cost equivalent to a network with $2n$ real nodes.
  
  About audio, it is true that a lot of analytics is done in Fourier/Laplace spaces, where complex numbers are mandatory. However, the incoming signal is real, also filters can be keep on this field. It is not usual, by example, the need of a DFT followed by a complex number NN.
  
  Finally, note one of the main advantages of a NN is their non-linear structure and capabilities. Usual transforms and related tools are restricted to linear. In this point, the NNs are a step after.
  
  
  ---
  
  