Answers.Body: <p>I found out the curvy zigzag green line is not polynomial as if it were polynomial, a vertical line won't cut that curvy line more than 1 time. </p>
  
  <p>It's the combination of straight lines (linear separation) of multiple neurons in the same layer. So it's linear separation ('linear' by previous_layer_output*weight, 'separation' by activation function), at multiple nodes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-13T10:31:15.020Z
Answers.LastActivityDate: 2019-09-13T10:31:15.020Z
Answers.UserDisplayName: Dan D.
Answers.Body: <p>What you have depicted is a nonlinear classificator. Although each stage does a linear separation, the sequential composition of linear separations is nonlinear. The nonlinearity of the neuron is key in this regard, as otherwise it would all be equivalent to a matrix multiplication, which is linear. You were right about the degree, although it's rarely called like that. People usually describe just the number of layers, and I guess the main reason is that it's not directly equivalent to such polynomials, as that depends on other factors (e.g. the activation function).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-11T07:26:47.480Z
Answers.LastActivityDate: 2019-09-11T07:26:47.480Z
Answers.UserDisplayName: David
Body: <p>A single neuron will be able to do linear separation. For example, XOR simulator network:</p>
  
  <pre><code>x1 --- n1.1
     \  /    \
      \/      \
               n2.1 
      /\      /
     /  \    /
  x2 --- n1.2 
  </code></pre>
  
  <p>Where <code>x1</code>, <code>x2</code> are the 2 inputs, <code>n1.1</code> and <code>n1.2</code> are the 2 neurons in hidden layer, and <code>n2.1</code> is the output neuron.</p>
  
  <p>The output neuron <code>n2.1</code> does a <strong>linear separation</strong>. How about the 2 neurons in hidden layer? </p>
  
  <p><strong>Is it still called linear separation</strong> (at 2 nodes and join the 2 separation lines)? or <strong>polynomial separation of degree 2</strong>?</p>
  
  <p>I'm confused about how it's called because there are curvy lines in this wiki article: <a href="https://en.wikipedia.org/wiki/Overfitting" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Overfitting</a></p>
  
  <p><a href="https://i.sstatic.net/o8idU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/o8idU.png" alt="Some curvy separation?"></a></p>
  
  <p><a href="https://i.sstatic.net/UByN2.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/UByN2.png" alt="Some other curvy separation?"></a></p>
  
Tags: <deep-learning><math><deep-neural-networks><artificial-neuron><linear-regression>
Title: Is it still called linear separation with a layer of more than 1 neuron
UserDisplayName: Dan D.
ThreadText: # Is it still called linear separation with a layer of more than 1 neuron
  
  (Asked by: username_0 on 2019-09-11)
  
  A single neuron will be able to do linear separation. For example, XOR simulator network:
  x1 --- n1.1
     \  /    \
      \/      \
               n2.1 
      /\      /
     /  \    /
  x2 --- n1.2 
  
  Where x1, x2 are the 2 inputs, n1.1 and n1.2 are the 2 neurons in hidden layer, and n2.1 is the output neuron.
  The output neuron n2.1 does a linear separation. How about the 2 neurons in hidden layer? 
  Is it still called linear separation (at 2 nodes and join the 2 separation lines)? or polynomial separation of degree 2?
  I'm confused about how it's called because there are curvy lines in this wiki article: https://en.wikipedia.org/wiki/Overfitting
  
  **Tags:** deep-learning, math, deep-neural-networks, artificial-neuron, linear-regression
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2019-09-13. Score: 0 | Accepted answer
  
  I found out the curvy zigzag green line is not polynomial as if it were polynomial, a vertical line won't cut that curvy line more than 1 time.
  
  It's the combination of straight lines (linear separation) of multiple neurons in the same layer. So it's linear separation ('linear' by previous\_layer\_output\*weight, 'separation' by activation function), at multiple nodes.
  
  
  ---
  
  ### Answer by username_1 on 2019-09-11. Score: 0
  
  What you have depicted is a nonlinear classificator. Although each stage does a linear separation, the sequential composition of linear separations is nonlinear. The nonlinearity of the neuron is key in this regard, as otherwise it would all be equivalent to a matrix multiplication, which is linear. You were right about the degree, although it's rarely called like that. People usually describe just the number of layers, and I guess the main reason is that it's not directly equivalent to such polynomials, as that depends on other factors (e.g. the activation function).
  
  
  ---
  
  