Body: <p>I am training a convLSTM with a dropout layer (with prob 0.5). </p>
  
  <p>If I train over more than 5 epochs I notice that the network starts to overfit: my validation set loss becomes stationary while the train loss keeps going down with every epoch. </p>
  
  <p>And if I train for 20 or more epochs the gap between the validation and train loss is quite substantial. At the same time precision-recall curve becomes much more stable (i.e. monotonic) if i train with a large number of epochs (e.g. 20). Why is that? Is this behaviour a common occurrence?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-11T14:14:19.427Z
Comments.Text: on the validation set
Comments.UserDisplayName: hellmean
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-11T14:54:21.153Z
Comments.Text: if validation loss does not bounce and start to get worse I would not call it overfitting...but what you are describing is common and not necessarily bad...the NN is memorizing the training examples, but if it is not impacting the validation loss adversely it might not be a bad thing...to get better validation loss you can try more/better data or regularization like L2.
Comments.UserDisplayName: j314erre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-12T14:05:37.853Z
Comments.Text: yes, the precision-recall curve for the model trained with 20 epochs. When you train with less epochs it can have multiple dips (local minima).
Comments.UserDisplayName: hellmean
Tags: <neural-networks><overfitting><performance>
Title: Why does precision-recall curve become more stable when neural net begins to overfit?
UserDisplayName: hellmean
ThreadText: # Why does precision-recall curve become more stable when neural net begins to overfit?
  
  (Asked by: username_0 on 2019-04-11)
  
  I am training a convLSTM with a dropout layer (with prob 0.5).
  
  If I train over more than 5 epochs I notice that the network starts to overfit: my validation set loss becomes stationary while the train loss keeps going down with every epoch.
  
  And if I train for 20 or more epochs the gap between the validation and train loss is quite substantial. At the same time precision-recall curve becomes much more stable (i.e. monotonic) if i train with a large number of epochs (e.g. 20). Why is that? Is this behaviour a common occurrence?
  
  **Tags:** neural-networks, overfitting, performance
  > **Comments:**
  > **username_0**: on the validation set  
  > **username_1**: if validation loss does not bounce and start to get worse I would not call it overfitting...but what you are describing is common and not necessarily bad...the NN is memorizing the training examples, but if it is not impacting the validation loss adversely it might not be a bad thing...to get better validation loss you can try more/better data or regularization like L2.  
  > **username_0**: yes, the precision-recall curve for the model trained with 20 epochs. When you train with less epochs it can have multiple dips (local minima).  
  
  ---
  
  ## 0 answers
  
  