Answers.Body: <p>Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily. </p>
  
  <p>The mathematical reason for the neural networks prowess at classifying is the <a href="https://en.wikipedia.org/wiki/Universal_approximation_theorem" rel="noreferrer">universal approximation theorem</a>. Which states that a neural network can approximate any continuous real-valued function on a compact subset. The quality of the approximation depends on the number of neurons. It has also been shown that adding the neurons in additional layers instead of adding them to existing layers improves the quality of the approximation faster. </p>
  
  <p>Add to that the not well-understood effectiveness of the <a href="https://en.wikipedia.org/wiki/Backpropagation" rel="noreferrer">backpropagation</a> algorithm and you have a setup then can actually learn the function that the UAT promises or something close. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-30T06:18:56.870Z
Answers.LastActivityDate: 2016-12-30T06:18:56.870Z
Answers.UserDisplayName: BlindKungFuMaster
Answers.Body: <p>With Neural Networks you simply classify datas. If you classify correctly, so you can do future classifications.</p>
  
  <p>How It Works?</p>
  
  <p>Simple neural networks like Perceptron can draw <strong>one</strong> decision boundary in order to classify datas.</p>
  
  <p>For example suppose you want to solve simple AND problem with simple Neural Network. You have 4 sample data containing x1 and x2 and weight vector containing w1 and w2. Suppose initial weight vector is [0 0]. If you made calculation which depend on NN algoritm. At the end, you should have a weight vector [1 1] or something like this.</p>
  
  <p><a href="https://i.sstatic.net/ELIum.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ELIum.jpg" alt="enter image description here"></a></p>
  
  <p>Please focus on the graphic. </p>
  
  <p>It says: I can classify input values into two classes (0 and 1). Ok. Then how can I do this? It is too simple. First sum input values (x1 and x2). </p>
  
  <blockquote>
    <p>0+0=0</p>
    
    <p>0+1=1</p>
    
    <p>1+0=1</p>
    
    <p>1+1=2</p>
  </blockquote>
  
  <p>It says:</p>
  
  <blockquote>
    <p>if sum&lt;1.5 then its class is 0</p>
    
    <p>if sum>1.5 then its class is 1</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-01-02T12:09:49.667Z
Answers.LastActivityDate: 2017-01-02T12:09:49.667Z
Answers.UserDisplayName: netizen
Answers.Body: <p>Neural networks excel at a variety of tasks, but to get an understanding of exactly why, it may be easier to take a particular task like classification and dive deeper.</p>
  
  <p>In simple terms, machine learning techniques learn a function to predict which class a particular input belongs to, depending on past examples. What sets neural nets apart is their ability to construct these functions that can explain even complex patterns in the data. The heart of a neural network is an activation function like Relu, which allows it to draw some basic classification boundaries like: <a href="https://i.sstatic.net/NmxyL.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/NmxyL.png" alt="Example classification boundaries for Relu"></a></p>
  
  <p>By composing hundreds of such Relus together, neural networks can create arbitrarily complex classification boundaries, for example:<a href="https://i.sstatic.net/vzPGD.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vzPGD.png" alt="enter image description here"></a></p>
  
  <p>In this article, I try to explain the intuition behind what makes neural networks work: <a href="https://medium.com/machine-intelligence-report/how-do-neural-networks-work-57d1ab5337ce" rel="nofollow noreferrer">https://medium.com/machine-intelligence-report/how-do-neural-networks-work-57d1ab5337ce</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-17T05:17:31.387Z
Answers.LastActivityDate: 2018-07-17T14:39:38.713Z
Answers.LastEditDate: 2018-07-17T14:39:38.713Z
Answers.UserDisplayName: Malay Haldar
Answers.Body: <p>In Neural Networks we consider everything in high dimension and try to find a hyperplane that classify them by small changes...</p>
  
  <p>Probably it is hard to prove that it works but intuition says if it can be classified you can do it by add a relaxed plane and let it move amongst data to find a local optimum... </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-01-01T17:02:33.327Z
Answers.LastActivityDate: 2017-01-01T17:02:33.327Z
Answers.UserDisplayName: MR_BD
Body: <p>I am new to neural-network and I am trying to understand mathematically what makes neural networks so good at classification problems. </p>
  
  <p>By taking the example of a small neural network (for example, one with 2 inputs, 2 nodes in a hidden layer and 2 nodes for the output), all you have is a complex function at the output which is mostly sigmoid over a linear combination of the sigmoid.</p>
  
  <p>So, how does that make them good at prediction? Does the final function lead to some sort of curve fitting?</p>
  
Tags: <neural-networks><classification><prediction>
Title: What makes neural networks so good at predictions?
UserDisplayName: Aditya Gupta
ThreadText: # What makes neural networks so good at predictions?
  
  (Asked by: username_0 on 2016-12-21)
  
  I am new to neural-network and I am trying to understand mathematically what makes neural networks so good at classification problems.
  
  By taking the example of a small neural network (for example, one with 2 inputs, 2 nodes in a hidden layer and 2 nodes for the output), all you have is a complex function at the output which is mostly sigmoid over a linear combination of the sigmoid.
  
  So, how does that make them good at prediction? Does the final function lead to some sort of curve fitting?
  
  **Tags:** neural-networks, classification, prediction
  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_1 on 2016-12-30. Score: 9
  
  Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily.
  
  The mathematical reason for the neural networks prowess at classifying is the [universal approximation theorem](https://en.wikipedia.org/wiki/Universal_approximation_theorem). Which states that a neural network can approximate any continuous real-valued function on a compact subset. The quality of the approximation depends on the number of neurons. It has also been shown that adding the neurons in additional layers instead of adding them to existing layers improves the quality of the approximation faster.
  
  Add to that the not well-understood effectiveness of the [backpropagation](https://en.wikipedia.org/wiki/Backpropagation) algorithm and you have a setup then can actually learn the function that the UAT promises or something close.
  
  
  ---
  
  ### Answer by username_2 on 2017-01-02. Score: 2
  
  With Neural Networks you simply classify datas. If you classify correctly, so you can do future classifications.
  
  How It Works?
  
  Simple neural networks like Perceptron can draw **one** decision boundary in order to classify datas.
  
  For example suppose you want to solve simple AND problem with simple Neural Network. You have 4 sample data containing x1 and x2 and weight vector containing w1 and w2. Suppose initial weight vector is [0 0]. If you made calculation which depend on NN algoritm. At the end, you should have a weight vector [1 1] or something like this.
  
  [![enter image description here](https://i.sstatic.net/ELIum.jpg)](https://i.sstatic.net/ELIum.jpg)
  
  Please focus on the graphic.
  
  It says: I can classify input values into two classes (0 and 1). Ok. Then how can I do this? It is too simple. First sum input values (x1 and x2).
  
  > 0+0=0
  >
  > 0+1=1
  >
  > 1+0=1
  >
  > 1+1=2
  
  It says:
  
  > if sum<1.5 then its class is 0
  >
  > if sum>1.5 then its class is 1
  
  
  ---
  
  ### Answer by username_3 on 2018-07-17. Score: 1
  
  Neural networks excel at a variety of tasks, but to get an understanding of exactly why, it may be easier to take a particular task like classification and dive deeper.
  
  In simple terms, machine learning techniques learn a function to predict which class a particular input belongs to, depending on past examples. What sets neural nets apart is their ability to construct these functions that can explain even complex patterns in the data. The heart of a neural network is an activation function like Relu, which allows it to draw some basic classification boundaries like: [![Example classification boundaries for Relu](https://i.sstatic.net/NmxyL.png)](https://i.sstatic.net/NmxyL.png)
  
  By composing hundreds of such Relus together, neural networks can create arbitrarily complex classification boundaries, for example:[![enter image description here](https://i.sstatic.net/vzPGD.png)](https://i.sstatic.net/vzPGD.png)
  
  In this article, I try to explain the intuition behind what makes neural networks work: <https://medium.com/machine-intelligence-report/how-do-neural-networks-work-57d1ab5337ce>
  
  
  ---
  
  ### Answer by username_4 on 2017-01-01. Score: 0
  
  In Neural Networks we consider everything in high dimension and try to find a hyperplane that classify them by small changes...
  
  Probably it is hard to prove that it works but intuition says if it can be classified you can do it by add a relaxed plane and let it move amongst data to find a local optimum...
  
  
  ---
  
  