Answers.Body: <p>This question has been asked a year ago when I faced this problem I searched and it hasn't any answers on it, I tried different ways and finally, data augmentation helped me.
  I used data augmentation and a very small learning rate. If the fluctuations are big, the batch size should be increased, and the learning rate should be decreased. After all, using more epochs helps you to have an almost smooth plot.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-07T07:54:08.357Z
Answers.LastActivityDate: 2020-11-07T07:54:08.357Z
Answers.UserDisplayName: SahaTib
Answers.Body: <p>That does not mean something really. While getting closer to the global minimum, your gradients fluctuates only. You just need to check the overall behaviour such if there is any over-fitting etc.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-30T15:35:30.650Z
Answers.LastActivityDate: 2022-07-30T15:35:30.650Z
Answers.UserDisplayName: ai-py
Body: <p>I was working on CNN. I modified the training procedure on runtime.</p>
  <p><a href="https://i.sstatic.net/0hrgz.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/0hrgz.png" alt="Validation Loss and Validation Accuracy" /></a></p>
  <p>As we can see from the validation loss and validation accuracy, the yellow curve does not fluctuate much. The green curve and red curve fluctuate suddenly to higher validation loss and lower validation accuracy, then goes to the lower validation loss and the higher validation accuracy, especially for the green curve.</p>
  <p>Is it happening because of overfitting or something else?</p>
  <p>I am asking it because, after fluctuation, the loss decreases to the lowest point, and also the accuracy increases to the highest point.</p>
  <p>Can anyone tell me why is it happening?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-03T06:45:46.717Z
Comments.Text: I have the same problem with my CNN training, Any ideas?
Comments.UserDisplayName: SahaTib
Tags: <neural-networks><deep-learning><data-preprocessing><data-augmentation>
Title: Validation Loss Fluctuates then Decrease alongside Validation Accuracy Increases
UserDisplayName: Mahir Mahbub
ThreadText: # Validation Loss Fluctuates then Decrease alongside Validation Accuracy Increases
  
  (Asked by: username_0 on 2019-06-10)
  
  I was working on CNN. I modified the training procedure on runtime.
  
  [![Validation Loss and Validation Accuracy](https://i.sstatic.net/0hrgz.png)](https://i.sstatic.net/0hrgz.png)
  
  As we can see from the validation loss and validation accuracy, the yellow curve does not fluctuate much. The green curve and red curve fluctuate suddenly to higher validation loss and lower validation accuracy, then goes to the lower validation loss and the higher validation accuracy, especially for the green curve.
  
  Is it happening because of overfitting or something else?
  
  I am asking it because, after fluctuation, the loss decreases to the lowest point, and also the accuracy increases to the highest point.
  
  Can anyone tell me why is it happening?
  
  **Tags:** neural-networks, deep-learning, data-preprocessing, data-augmentation
  
  > **Comments:**
  > **username_1**: I have the same problem with my CNN training, Any ideas?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-11-07. Score: 0
  
  This question has been asked a year ago when I faced this problem I searched and it hasn't any answers on it, I tried different ways and finally, data augmentation helped me.
  I used data augmentation and a very small learning rate. If the fluctuations are big, the batch size should be increased, and the learning rate should be decreased. After all, using more epochs helps you to have an almost smooth plot.
  
  
  ---
  
  ### Answer by username_2 on 2022-07-30. Score: 0
  
  That does not mean something really. While getting closer to the global minimum, your gradients fluctuates only. You just need to check the overall behaviour such if there is any over-fitting etc.
  
  
  ---
  
  