Answers.Body: <p>You are not training the model to give the optimum result for one input; You want the model to produce the minimum loss for the whole population of samples that model may be given. The inputs are only considered samples from that population. The more samples you have, if they are drawn i.i.d., the better they can represent that population.</p>
  <p>That is, you do not want a cat detector that is good at identifying one cat; You want a cat detector that can identify all cats.</p>
  <p>Also, data is often noisy. So, just relying on one sample won't lead you to the exact local minimum for the loss even for that particular sample. That is, if you collect that exact same sample again, the noise from the sensors may be different. So, the negative gradient of one sample may not point exactly towards the local minimum for the loss. By combining several samples, you can &quot;average out&quot; the errors and get closer to the actual minimum.</p>
  <p>Moreover, even if the gradient was pointing in the correct direction, you can still overshoot. Again, combining several different samples can help you minimize loss..</p>
  <p>I think, before reading about backpropagation, you need to understand the underlying theory. So, look up gradient descent and stochastic gradient descent. Preferably, you should have some understanding of calculus and statistics before you do that.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-17T16:46:43.033Z
Answers.Text: So, if I understand it right, what you mean is to use "the variants of all errors" instead of "error of a single input"?
Answers.UserDisplayName: Noob002
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-17T17:58:41.343Z
Answers.Text: @Noob002 that's one way to think about it. Another is that the optimum loss for "cats seen from the side"-classifier is probably not the same as for "cats seen from any angle"-classifier
Answers.UserDisplayName: Avatrin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-17T16:06:34.767Z
Answers.LastActivityDate: 2021-11-17T16:06:34.767Z
Answers.UserDisplayName: Avatrin
Body: <p>I often heard people saying, &quot;large set of training data is needed for producing an accurate AI&quot;.</p>
  <p>But when I looked for articles explaining backpropagations online, it all seems like you could get the job done by &quot;one single set of input&quot;, as long as you repeat the process enough times.</p>
  <p>So what's the &quot;large set of training data&quot; for!?</p>
  <p>After the optimized set of weights was calculated from the first input, plug in the second input and repeat the process again?</p>
  <p>Won't that &quot;screw up&quot; the result from the first input since it was &quot;tailored&quot; from it?</p>
  
Tags: <neural-networks><backpropagation><training-datasets>
Title: Why "large set of training data" is needed in Neural Network AI training?
UserDisplayName: Noob002
ThreadText: # Why "large set of training data" is needed in Neural Network AI training?
  
  (Asked by: username_0 on 2021-11-17)
  
  I often heard people saying, "large set of training data is needed for producing an accurate AI".
  
  But when I looked for articles explaining backpropagations online, it all seems like you could get the job done by "one single set of input", as long as you repeat the process enough times.
  
  So what's the "large set of training data" for!?
  
  After the optimized set of weights was calculated from the first input, plug in the second input and repeat the process again?
  
  Won't that "screw up" the result from the first input since it was "tailored" from it?
  
  **Tags:** neural-networks, backpropagation, training-datasets
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-11-17. Score: 3 | Accepted answer
  
  You are not training the model to give the optimum result for one input; You want the model to produce the minimum loss for the whole population of samples that model may be given. The inputs are only considered samples from that population. The more samples you have, if they are drawn i.i.d., the better they can represent that population.
  
  That is, you do not want a cat detector that is good at identifying one cat; You want a cat detector that can identify all cats.
  
  Also, data is often noisy. So, just relying on one sample won't lead you to the exact local minimum for the loss even for that particular sample. That is, if you collect that exact same sample again, the noise from the sensors may be different. So, the negative gradient of one sample may not point exactly towards the local minimum for the loss. By combining several samples, you can "average out" the errors and get closer to the actual minimum.
  
  Moreover, even if the gradient was pointing in the correct direction, you can still overshoot. Again, combining several different samples can help you minimize loss..
  
  I think, before reading about backpropagation, you need to understand the underlying theory. So, look up gradient descent and stochastic gradient descent. Preferably, you should have some understanding of calculus and statistics before you do that.
  
  > **Comments:**
  > **username_0**: So, if I understand it right, what you mean is to use "the variants of all errors" instead of "error of a single input"?  
  > **username_1**: @Noob002 that's one way to think about it. Another is that the optimum loss for "cats seen from the side"-classifier is probably not the same as for "cats seen from any angle"-classifier  
  
  ---
  
  