Answers.Body: <p>There are tons of &quot;neurons&quot; as @CuCaRot has pointed out. Each neuron type will have its corresponding operation to get the neuron output. We have Convolutional, Recurrent, Polynomial, Transformers...</p>
  <p>But to answer your specific question we usually do <code>Multiply + Add</code> function because they are highly optimized in dedicated hardware. That is why all sort of complex neurons tends to be split up in <code>Multiply + Add</code> modules.</p>
  <p>When you start worrying about network efficiency you will start to see the term FLOPs (Floating Point Operations) and MACs (Multiply Add Cumulate) operations. Because in some hardware the MACs operations can be executed very fast counting as if it were a single operation. More info <a href="https://ai.stackexchange.com/questions/23482/are-mult-adds-and-flops-equivalent#:%7E:text=According%20to%20this%20source%2C%20one,two%20FLOP%20(multiply%20accumulate).">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-04-21T12:14:16.070Z
Answers.Text: Also worth mentioning the universal approximation theorem, that in this context says that combining simple linear operations with a non-linear activation and combining into at least 2 layers is already complex enough to get the job done. Further complexity needs to be justified - perhaps because it directly models something known to be useful.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-22T11:51:42.063Z
Answers.LastActivityDate: 2022-03-22T11:51:42.063Z
Answers.UserDisplayName: JVGD
Body: <p>I have just started learning about NN and DL and I wanted to know if there is a theoretical reason we use a weighted sum for all the inputs in an artificial neuron. So for example if we have a neuron with two inputs which have weight <span class="math-container">$w_1$</span> and <span class="math-container">$w_2$</span>, why don't we use a function like <span class="math-container">$x_1^2\times w_1 + x_2\times w_2^2$</span>?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-22T10:38:22.253Z
Comments.Text: We do. its name is [polynomial equation](https://www.javatpoint.com/machine-learning-polynomial-regression)
Comments.UserDisplayName: CuCaRot
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-22T12:06:06.843Z
Comments.Text: Please check the explanation given in the link: https://stats.stackexchange.com/questions/291680/can-any-one-explain-why-dot-product-is-used-in-neural-network-and-what-is-the-in
Comments.UserDisplayName: le4m
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-22T12:11:39.140Z
Comments.Text: Can you please edit your post to format the math symbols and equations with mathjax, to improve the readability?
Comments.UserDisplayName: nbro
Tags: <neural-networks><artificial-neuron>
Title: Why do we use a weighted sum in an artificial neuron instead of another more complex function?
UserDisplayName: Nikkal
ThreadText: # Why do we use a weighted sum in an artificial neuron instead of another more complex function?
  
  (Asked by: username_0 on 2022-03-22)
  
  I have just started learning about NN and DL and I wanted to know if there is a theoretical reason we use a weighted sum for all the inputs in an artificial neuron. So for example if we have a neuron with two inputs which have weight $w\_1$ and $w\_2$, why don't we use a function like $x\_1^2\times w\_1 + x\_2\times w\_2^2$?
  
  **Tags:** neural-networks, artificial-neuron
  > **Comments:**
  > **username_1**: We do. its name is [polynomial equation](https://www.javatpoint.com/machine-learning-polynomial-regression)  
  > **username_2**: Please check the explanation given in the link: https://stats.stackexchange.com/questions/291680/can-any-one-explain-why-dot-product-is-used-in-neural-network-and-what-is-the-in  
  > **username_3**: Can you please edit your post to format the math symbols and equations with mathjax, to improve the readability?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_4 on 2022-03-22. Score: 3
  
  There are tons of "neurons" as @CuCaRot has pointed out. Each neuron type will have its corresponding operation to get the neuron output. We have Convolutional, Recurrent, Polynomial, Transformers...
  
  But to answer your specific question we usually do `Multiply + Add` function because they are highly optimized in dedicated hardware. That is why all sort of complex neurons tends to be split up in `Multiply + Add` modules.
  
  When you start worrying about network efficiency you will start to see the term FLOPs (Floating Point Operations) and MACs (Multiply Add Cumulate) operations. Because in some hardware the MACs operations can be executed very fast counting as if it were a single operation. More info [here](https://ai.stackexchange.com/questions/23482/are-mult-adds-and-flops-equivalent#:%7E:text=According%20to%20this%20source%2C%20one,two%20FLOP%20(multiply%20accumulate).).
  
  > **Comments:**
  > **username_5**: Also worth mentioning the universal approximation theorem, that in this context says that combining simple linear operations with a non-linear activation and combining into at least 2 layers is already complex enough to get the job done. Further complexity needs to be justified - perhaps because it directly models something known to be useful.  
  
  ---
  
  