Answers.Body: <p>Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-19T20:27:04.710Z
Answers.LastActivityDate: 2018-08-19T20:27:04.710Z
Answers.UserDisplayName: Mr. Eivind
Body: <p>As I know, a single layer neural network can only do linear operations, but multilayered ones can.</p>
  
  <p>Also, I recently learned that finite matrices/tensors, which are used in many neural networks, can <em>only represent linear</em> operations.</p>
  
  <p>However, multi-layered neural networks can represent non-linear (even much more complex than being just a nonlinear) operations.</p>
  
  <p>What makes it happen? The activation layer?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-08-19T19:45:26.750Z
Comments.Text: The very quick answer is; yes, nonlinear **activation functions** in between linear transformations (matrix multiplications) allow for the "total" to represent non-linear functions.
Comments.UserDisplayName: Dennis Soemers
Tags: <neural-networks><machine-learning><math><activation-functions><function-approximation>
Title: What makes multi-layer neural networks able to perform nonlinear operations?
UserDisplayName: Hyeonseo Yang
ThreadText: # What makes multi-layer neural networks able to perform nonlinear operations?
  
  (Asked by: username_0 on 2018-08-19)
  
  As I know, a single layer neural network can only do linear operations, but multilayered ones can.
  
  Also, I recently learned that finite matrices/tensors, which are used in many neural networks, can *only represent linear* operations.
  
  However, multi-layered neural networks can represent non-linear (even much more complex than being just a nonlinear) operations.
  
  What makes it happen? The activation layer?
  
  **Tags:** neural-networks, machine-learning, math, activation-functions, function-approximation
  > **Comments:**
  > **username_1**: The very quick answer is; yes, nonlinear **activation functions** in between linear transformations (matrix multiplications) allow for the "total" to represent non-linear functions.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-08-19. Score: 5 | Accepted answer
  
  Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.
  
  
  ---
  
  