Body: <p>I've been reading about different types of neurons that the traditional linear one. One example I came across is the Sigma-Pi neuron, where the activation function includes higher order terms, such as</p>
  <p><a href="https://i.sstatic.net/OIW47.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/OIW47.png" alt="<span class="math-container">$f$</span>" /></a></p>
  <p>where the activation has the additional higher order term on the right, as well as the standard term on the left. What are the advantages and disadvantages of using neurons like that?  Why do we tend not to use them?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-27T21:06:32.410Z
Comments.Text: Can you please 1. put your **specific** question in the title, 2. format your question with mathjax, 3. provide a link to a reference that talks about this "sigma-pi neuron" (for people that are not familiar with it)?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-28T13:07:32.647Z
Comments.Text: My question isn't specific to the sigma-pi neural network, but here is a link to that anyway: https://repository.up.ac.za/bitstream/handle/2263/29715/03chapter3.pdf?sequence=4
  
  The question at heart is fairly straightforward: why are we using linear activations of the form wX+c and not higher order activations? What advantages and disadvantages are there?
Comments.UserDisplayName: burn_burn_55
Tags: <neural-networks><activation-functions><artificial-neuron>
Title: What are the advantages and disadvantages of higher order neuron activation functions?
UserDisplayName: burn_burn_55
ThreadText: # What are the advantages and disadvantages of higher order neuron activation functions?
  
  (Asked by: username_0 on 2022-03-27)
  
  I've been reading about different types of neurons that the traditional linear one. One example I came across is the Sigma-Pi neuron, where the activation function includes higher order terms, such as
  
  [![<span class=](https://i.sstatic.net/OIW47.png)$f$" />](https://i.sstatic.net/OIW47.png)
  
  where the activation has the additional higher order term on the right, as well as the standard term on the left. What are the advantages and disadvantages of using neurons like that? Why do we tend not to use them?
  
  **Tags:** neural-networks, activation-functions, artificial-neuron
  > **Comments:**
  > **username_1**: Can you please 1. put your **specific** question in the title, 2. format your question with mathjax, 3. provide a link to a reference that talks about this "sigma-pi neuron" (for people that are not familiar with it)?  
  > **username_0**: My question isn't specific to the sigma-pi neural network, but here is a link to that anyway: https://repository.up.ac.za/bitstream/handle/2263/29715/03chapter3.pdf?sequence=4  The question at heart is fairly straightforward: why are we using linear activations of the form wX+c and not higher order activations? What advantages and disadvantages are there?  
  
  ---
  
  ## 0 answers
  
  