Answers.Body: <p>A quick scan of your plots does not seem to indicate any severe over fitting. As pointed out there is always some degree of over fitting but in this case it looks to be very small. Your validation loss reduces as it should down to what appears to be a very small level and remains low.
  One test would be to add a "dropout" layer into your model right after a dense layer and see the effect on training accuracy and validation accuracy. Set the drop out rate to something like .4 . Make sure your training accuracy remains high (it may take a few more epochs to get there) then look to see if the validation loss is lower than without the dropout layer. Run this several times because random weight initialization can sometimes effect accuracy by converging on a non optimal local minimum.
  Additional you can add kernel regularizes to your dense layers which also helps to prevent over training. I have a lot of plots similar to yours and adding dropout and regularization had no effect on validation accuracy.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-02T20:56:35.110Z
Answers.LastActivityDate: 2020-03-02T20:56:35.110Z
Answers.UserDisplayName: Gerry P
Answers.Body: <p>Overfitting nearly always occurs to some degree when fitting to limited data sets, and neural networks are very prone to it. However neither of your graphs show a major problem with overfitting - that is usually obvious when epoch counts increase, the results on training data continue to improve whilst the results on cross validation get progressively worse. Your validation results do not do that, and appear to remain stable.</p>
  
  <p>It is usually pragmatic to accept that there will be at least <em>some</em> difference between measurements on the training set and cross validation or test sets. The primary goal is usually to get the best measurements in test that you can. With that in mind, you are usually only interested in how much you are overfitting if it implies you could improve performance by using techniques to reduce overfitting e.g. various forms of regularisation.</p>
  
  <p>Without knowing your data set or known good results, it is hard to tell whether the difference you are seeing between test and train in accuracy could be improved. Your accuracy graph shows a train accuracy close to 100% and a validation accuracy close to ~96%. It looks a bit like MNIST results, and if I saw that result on MNIST I would suspect <em>something</em> was wrong, and it might be fixed by looking at regularisation (but it might also be something esle). However, that's only because I know that 99.7% accuracy is possible in test - on other problems I might be very happy with 96% accuracy.</p>
  
  <p>The loss graph is not very useful, since the scale has completely lost any difference there might be between training and validation. You should probably re-scale it to show detail close to 0 loss, and ignore the earlier large loss values.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-25T23:37:38.270Z
Answers.Text: It is about kind of binary image classification data, I am using an alexnet, regularization caused a lot of fluctuations in my model. I will add a sample of my data
Answers.UserDisplayName: SahaTib
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-25T22:36:44.690Z
Answers.LastActivityDate: 2020-02-25T22:36:44.690Z
Answers.UserDisplayName: Neil Slater
Body: <p>I am training a neural network and plot model accuracy and model loss.
  I am a little confused about overfitting. Is my model overfitted or not? how can I interpret it
  <a href="https://i.sstatic.net/m7sJ1.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/m7sJ1.png" alt="enter image description here"></a>
  <a href="https://i.sstatic.net/PKNwi.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PKNwi.png" alt="enter image description here"></a></p>
  
  <hr>
  
  <p><strong>EDIT:</strong> here is a sample of my input data, I have a binary image classification
  <a href="https://i.sstatic.net/kCoxz.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kCoxz.jpg" alt="enter image description here"></a></p>
  
Tags: <neural-networks><training>
Title: Is this model overfitted or not?
UserDisplayName: SahaTib
ThreadText: # Is this model overfitted or not?
  
  (Asked by: username_0 on 2020-02-25)
  
  I am training a neural network and plot model accuracy and model loss.
  I am a little confused about overfitting. Is my model overfitted or not? how can I interpret it
  [![enter image description here](https://i.sstatic.net/m7sJ1.png)](https://i.sstatic.net/m7sJ1.png)
  [![enter image description here](https://i.sstatic.net/PKNwi.png)](https://i.sstatic.net/PKNwi.png)
  
  ---
  
  **EDIT:** here is a sample of my input data, I have a binary image classification
  [![enter image description here](https://i.sstatic.net/kCoxz.jpg)](https://i.sstatic.net/kCoxz.jpg)
  
  **Tags:** neural-networks, training
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-03-02. Score: 1 | Accepted answer
  
  A quick scan of your plots does not seem to indicate any severe over fitting. As pointed out there is always some degree of over fitting but in this case it looks to be very small. Your validation loss reduces as it should down to what appears to be a very small level and remains low.
  One test would be to add a "dropout" layer into your model right after a dense layer and see the effect on training accuracy and validation accuracy. Set the drop out rate to something like .4 . Make sure your training accuracy remains high (it may take a few more epochs to get there) then look to see if the validation loss is lower than without the dropout layer. Run this several times because random weight initialization can sometimes effect accuracy by converging on a non optimal local minimum.
  Additional you can add kernel regularizes to your dense layers which also helps to prevent over training. I have a lot of plots similar to yours and adding dropout and regularization had no effect on validation accuracy.
  
  
  ---
  
  ### Answer by username_2 on 2020-02-25. Score: 3
  
  Overfitting nearly always occurs to some degree when fitting to limited data sets, and neural networks are very prone to it. However neither of your graphs show a major problem with overfitting - that is usually obvious when epoch counts increase, the results on training data continue to improve whilst the results on cross validation get progressively worse. Your validation results do not do that, and appear to remain stable.
  
  It is usually pragmatic to accept that there will be at least *some* difference between measurements on the training set and cross validation or test sets. The primary goal is usually to get the best measurements in test that you can. With that in mind, you are usually only interested in how much you are overfitting if it implies you could improve performance by using techniques to reduce overfitting e.g. various forms of regularisation.
  
  Without knowing your data set or known good results, it is hard to tell whether the difference you are seeing between test and train in accuracy could be improved. Your accuracy graph shows a train accuracy close to 100% and a validation accuracy close to ~96%. It looks a bit like MNIST results, and if I saw that result on MNIST I would suspect *something* was wrong, and it might be fixed by looking at regularisation (but it might also be something esle). However, that's only because I know that 99.7% accuracy is possible in test - on other problems I might be very happy with 96% accuracy.
  
  The loss graph is not very useful, since the scale has completely lost any difference there might be between training and validation. You should probably re-scale it to show detail close to 0 loss, and ignore the earlier large loss values.
  
  > **Comments:**
  > **username_0**: It is about kind of binary image classification data, I am using an alexnet, regularization caused a lot of fluctuations in my model. I will add a sample of my data  
  
  ---
  
  