Answers.Body: <p>Validation results will almost never be as good as training results; that's just natural. As long as they are not <em>too</em> different, you should be fine. What &quot;too different&quot; means depends on the particular data set and model you're using.</p>
  <p>If you plot the curves for varying parameter values, when the training error keeps going down but the validation error starts going up again, that's when you know there is overfitting. In your second graph, after 14 epochs, we might see the start of overfitting. If you continue this until 20 epochs or so, it should be even more clear. I would guess that 12 is probably a good value for the number of epochs for that problem.</p>
  <p>In the first graph, we don't see that happening yet. The model might not be well-suited (the gap between training and validation results is a bit larger)  but that can also be because of too little data, or other factors. Perhaps that's just the best you can do; there might be noise in the data or something.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-22T01:49:47.883Z
Answers.LastActivityDate: 2020-09-22T01:49:47.883Z
Answers.UserDisplayName: Robby Goetschalckx
Body: <p>I have a deep learning configuration in which I obtain good results on the validation set but even better results in the training set. From my understanding this means that there is overfitting to some extent. What does this mean in practice? Does it mean that my model is not good and that I should not use it? If I decrease the gap between the validation and training accuracy (decreasing the overfitting) but at the same time decrease the validation accuracy, which of the two models is better?</p>
  <p>Below are some images to illustrate the two situations outlined previously:</p>
  <p><a href="https://i.sstatic.net/moq2x.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/moq2x.png" alt="High validation accuracy and even higher training accuracy " /></a></p>
  <p><a href="https://i.sstatic.net/cCWmy.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/cCWmy.png" alt="Smaller gap between validation and training accuracy" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-22T07:26:28.147Z
Comments.Text: Higher validation is almost always better. If you think about it, the best network in regards to keeping the difference between training and validation is one that's terrible, because it will perform equally bad on both. If you're validation accuracy is increasing, you can be confident that if the network is over-fitting to the training data, it isn't to the point where it impacts performance. There will always be some level of over-fitting, but if validation accuracy continues to increase, that's ok.
Comments.UserDisplayName: Recessive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-24T16:24:13.883Z
Comments.Text: This is **not** overfitting, but the (always expected, in some degree) [generalization gap](https://ai.googleblog.com/2019/07/predicting-generalization-gap-in-deep.html).
Comments.UserDisplayName: desertnaut
Tags: <machine-learning><overfitting>
Title: How much overfitting is acceptable?
UserDisplayName: Tony Reis
ThreadText: # How much overfitting is acceptable?
  
  (Asked by: username_0 on 2020-09-21)
  
  I have a deep learning configuration in which I obtain good results on the validation set but even better results in the training set. From my understanding this means that there is overfitting to some extent. What does this mean in practice? Does it mean that my model is not good and that I should not use it? If I decrease the gap between the validation and training accuracy (decreasing the overfitting) but at the same time decrease the validation accuracy, which of the two models is better?
  
  Below are some images to illustrate the two situations outlined previously:
  
  [![High validation accuracy and even higher training accuracy ](https://i.sstatic.net/moq2x.png)](https://i.sstatic.net/moq2x.png)
  
  [![Smaller gap between validation and training accuracy](https://i.sstatic.net/cCWmy.png)](https://i.sstatic.net/cCWmy.png)
  
  **Tags:** machine-learning, overfitting
  
  > **Comments:**
  > **username_1**: Higher validation is almost always better. If you think about it, the best network in regards to keeping the difference between training and validation is one that's terrible, because it will perform equally bad on both. If you're validation accuracy is increasing, you can be confident that if the network is over-fitting to the training data, it isn't to the point where it impacts performance. There will always be some level of over-fitting, but if validation accuracy continues to increase, that's ok.  
  > **username_2**: This is **not** overfitting, but the (always expected, in some degree) [generalization gap](https://ai.googleblog.com/2019/07/predicting-generalization-gap-in-deep.html).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2020-09-22. Score: 1
  
  Validation results will almost never be as good as training results; that's just natural. As long as they are not *too* different, you should be fine. What "too different" means depends on the particular data set and model you're using.
  
  If you plot the curves for varying parameter values, when the training error keeps going down but the validation error starts going up again, that's when you know there is overfitting. In your second graph, after 14 epochs, we might see the start of overfitting. If you continue this until 20 epochs or so, it should be even more clear. I would guess that 12 is probably a good value for the number of epochs for that problem.
  
  In the first graph, we don't see that happening yet. The model might not be well-suited (the gap between training and validation results is a bit larger) but that can also be because of too little data, or other factors. Perhaps that's just the best you can do; there might be noise in the data or something.
  
  
  ---
  
  