Answers.Body: <p>When we add a single layer with a non-linear activation function, right after the application of the activation function, a new basis function is found (for that neuron), which is some combination of the weights and biases, which acts as a new way to view or analyze the feature sets. </p>
  
  <p>With an increasingly deeper network, we keep finding representations, which are new basis vectors of the combination of the previous layer's weights and biases, that is, higher-level representations. </p>
  
  <p>If they're error-free, they'll give better performances, but if small errors creep in earlier basis vectors, the error increases through depth.</p>
  
  <p>A nice analogy is the Taylor series, where <span class="math-container">$1$</span>, <span class="math-container">$x$</span>, <span class="math-container">$x^2$</span>, and so on, are the basis vectors for estimating the function in 1D.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-23T10:54:16.193Z
Answers.LastActivityDate: 2020-03-11T00:36:36.360Z
Answers.LastEditDate: 2020-03-11T00:36:36.360Z
Answers.UserDisplayName: sagar_acharya
Body: <p>I'm learning about multilayer perceptrons, and I have a quick theory question in regards to hidden layer neurons. </p>
  
  <p>I know we can use two hidden layers to solve a non-linearly separable problem by allowing for a representation with two linear separators. However, we can solve a non-linearly separable problem using <strong><em>only one</em></strong> hidden layer. </p>
  
  <p>This seems fine, but what kind of representation does one hidden layer add? And how is the output of the network affected?</p>
  
  <p>I've drawn a diagram of a multilayer perceptron with one hidden layer neuron. I used this same layout to solve a non-linearly separable problem. The single hidden layer node is inside the red square. Forgive my poor MS-Paint skills.</p>
  
  <p><a href="https://i.sstatic.net/qGm17.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/qGm17.png" alt="simple perceptron"></a></p>
  
Tags: <neural-networks><machine-learning><hidden-layers><multilayer-perceptrons>
Title: How does a single hidden layer affect output?
UserDisplayName: Howard P
ThreadText: # How does a single hidden layer affect output?
  
  (Asked by: username_0 on 2018-10-08)
  
  I'm learning about multilayer perceptrons, and I have a quick theory question in regards to hidden layer neurons.
  
  I know we can use two hidden layers to solve a non-linearly separable problem by allowing for a representation with two linear separators. However, we can solve a non-linearly separable problem using ***only one*** hidden layer.
  
  This seems fine, but what kind of representation does one hidden layer add? And how is the output of the network affected?
  
  I've drawn a diagram of a multilayer perceptron with one hidden layer neuron. I used this same layout to solve a non-linearly separable problem. The single hidden layer node is inside the red square. Forgive my poor MS-Paint skills.
  
  [![simple perceptron](https://i.sstatic.net/qGm17.png)](https://i.sstatic.net/qGm17.png)
  
  **Tags:** neural-networks, machine-learning, hidden-layers, multilayer-perceptrons
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-10-23. Score: 1 | Accepted answer
  
  When we add a single layer with a non-linear activation function, right after the application of the activation function, a new basis function is found (for that neuron), which is some combination of the weights and biases, which acts as a new way to view or analyze the feature sets.
  
  With an increasingly deeper network, we keep finding representations, which are new basis vectors of the combination of the previous layer's weights and biases, that is, higher-level representations.
  
  If they're error-free, they'll give better performances, but if small errors creep in earlier basis vectors, the error increases through depth.
  
  A nice analogy is the Taylor series, where $1$, $x$, $x^2$, and so on, are the basis vectors for estimating the function in 1D.
  
  
  ---
  
  