Answers.Body: <p>Welcome to AI.SE @bilanush.</p>
  
  <p>Here's an example approach that might make things clearer. There are other ways to train a neural network to do this however.</p>
  
  <p>In your earlier example with an image, you probably noticed that the network receives the image as a series of values, representing each pixel in the image. The network then learns which of a series of output neurons should be active in response to a given set of pixel values. Those output neurons, when read in an appropriate way, correspond to the correct label for the image. The difference between the set of outputs that should have been active, and the set that were active, forms the basis of the error signal that allows the network to learn.</p>
  
  <p>You've probably heard that computers represent numbers with <em>binary digits</em>. So you could think of the number 16 as being: 00010000 in "8-bit binary". In 16-bit binary, this number would be 0000000000010000, and so on.</p>
  
  <p>So one way of viewing your problem is a function mapping binary inputs to binary outputs (very similar to labelling a black-and-white image). For instance, the input 00010000 (16) should produce the output 00010001 (17). The input 00100011 (35) should produce the output 00100100 (36), and so on.</p>
  
  <p>As before, you will have a set of output neurons. In this case, it should be wide as the set of input neurons. As before, the error signal is the difference between the expected inputs and outputs.</p>
  
  <p>As to the question of how they can learn this function "without plus", in fact the individual neurons in a network perform just two operations: addition of their inputs, and a non-linear transformation of the sum. It has been proven that these are sufficient to learn any function from inputs to outputs, as long as the network contains 3 layers or more, and the as long as the middle layer is wide enough, but here it should be easy to see how addition might emerge.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-28T01:45:29.720Z
Answers.Text: @DuttaA I think you are mistaken: a sufficiently wide and deep network can learn this pattern and extrapolate to unseen examples. In particular, since the pattern is a series of full-adder gates, it can be done simply with a network that has depth and width proportionate to the number of bits. I agree with your answer that there are better approaches to solving this problem, but they may be harder for a novice to follow.
Answers.UserDisplayName: John Doucette
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-27T12:27:25.177Z
Answers.LastActivityDate: 2018-08-27T12:27:25.177Z
Answers.UserDisplayName: John Doucette
Body: <p>I am trying to understant how it works.  How do you teach it say, to add 1 to each number it gets.  I am pretty new to the subject and I learned how it works when you teach it to identify a picture of a number.  I can understand how it identifies a number but I cant get it how would it study to perform addition? I can understand that it can  identify a number or picture using the pixels and assigning weights and then learning to measure whether a picture of a number resembling the weight is assigned to each pixel. But i can't logically understand how would it learn the concept  of adding a number by one.  Suppose I showed it thousands of examples of 7 turning to 8 152 turning into 153 would it get it that every number in the world has to be added by one? How would it get it having no such operation of + ? Since addition does not exist to its proposal then how can it realize that it has to add one in every number? Even by seeing thousands of examples but having no such operation of plus I cant understand it.    I could understand identifying pixels and such but such an operation I cant get the theoretical logic behind it. Can you explain the logic in layman terms?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-08-27T11:51:45.857Z
Comments.Text: "having no such operation of plus..." why do you say this? Neural networks perform linear combinations of inputs, followed by non-linearities. Of course they have the + operation!
Comments.UserDisplayName: firion
Tags: <neural-networks><deep-learning>
Title: How is it possible to teach a neural network to perform addition?
UserDisplayName: bilanush
ThreadText: # How is it possible to teach a neural network to perform addition?
  
  (Asked by: username_0 on 2018-08-27)
  
  I am trying to understant how it works. How do you teach it say, to add 1 to each number it gets. I am pretty new to the subject and I learned how it works when you teach it to identify a picture of a number. I can understand how it identifies a number but I cant get it how would it study to perform addition? I can understand that it can identify a number or picture using the pixels and assigning weights and then learning to measure whether a picture of a number resembling the weight is assigned to each pixel. But i can't logically understand how would it learn the concept of adding a number by one. Suppose I showed it thousands of examples of 7 turning to 8 152 turning into 153 would it get it that every number in the world has to be added by one? How would it get it having no such operation of + ? Since addition does not exist to its proposal then how can it realize that it has to add one in every number? Even by seeing thousands of examples but having no such operation of plus I cant understand it. I could understand identifying pixels and such but such an operation I cant get the theoretical logic behind it. Can you explain the logic in layman terms?
  
  **Tags:** neural-networks, deep-learning
  > **Comments:**
  > **username_1**: "having no such operation of plus..." why do you say this? Neural networks perform linear combinations of inputs, followed by non-linearities. Of course they have the + operation!  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-08-27. Score: 1
  
  Welcome to AI.SE @bilanush.
  
  Here's an example approach that might make things clearer. There are other ways to train a neural network to do this however.
  
  In your earlier example with an image, you probably noticed that the network receives the image as a series of values, representing each pixel in the image. The network then learns which of a series of output neurons should be active in response to a given set of pixel values. Those output neurons, when read in an appropriate way, correspond to the correct label for the image. The difference between the set of outputs that should have been active, and the set that were active, forms the basis of the error signal that allows the network to learn.
  
  You've probably heard that computers represent numbers with *binary digits*. So you could think of the number 16 as being: 00010000 in "8-bit binary". In 16-bit binary, this number would be 0000000000010000, and so on.
  
  So one way of viewing your problem is a function mapping binary inputs to binary outputs (very similar to labelling a black-and-white image). For instance, the input 00010000 (16) should produce the output 00010001 (17). The input 00100011 (35) should produce the output 00100100 (36), and so on.
  
  As before, you will have a set of output neurons. In this case, it should be wide as the set of input neurons. As before, the error signal is the difference between the expected inputs and outputs.
  
  As to the question of how they can learn this function "without plus", in fact the individual neurons in a network perform just two operations: addition of their inputs, and a non-linear transformation of the sum. It has been proven that these are sufficient to learn any function from inputs to outputs, as long as the network contains 3 layers or more, and the as long as the middle layer is wide enough, but here it should be easy to see how addition might emerge.
  
  > **Comments:**
  > **username_2**: @DuttaA I think you are mistaken: a sufficiently wide and deep network can learn this pattern and extrapolate to unseen examples. In particular, since the pattern is a series of full-adder gates, it can be done simply with a network that has depth and width proportionate to the number of bits. I agree with your answer that there are better approaches to solving this problem, but they may be harder for a novice to follow.  
  
  ---
  
  