Answers.Body: <p>Without more information the best diagnostic is:</p>
  <p><em>You potentially have a bug in your code.</em></p>
  <p><strong>Justification</strong></p>
  <p>Neural network systems are capable of failing silently. The system can still appear to &quot;learn&quot; even in the presence of said bug - a while back I had a very similar issue with a toy CNN project. My network could get 99% accuracy on the training set but always achieved 8-13% on the validation set. This looked like overfit but none of the methods solved the issue. Finally, I found that I wasn't feeding the data correctly into the network during train time but I <strong>was</strong> feeding the data correctly during validation time.</p>
  <p><strong>Conclusion</strong></p>
  <p>Provide the following for better diagnostic:</p>
  <ul>
  <li>Is this a custom CNN you hardcoded?</li>
  <li>are you using python?</li>
  <li>what's the objective function?</li>
  <li>can you provide the loss curve for the training set?</li>
  <li>show us some code too (this will be very helpful for working toward a solution)</li>
  </ul>
  <p>I hope this helps, and best of luck!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-01T17:44:47.027Z
Answers.LastActivityDate: 2020-07-25T16:14:12.170Z
Answers.LastEditDate: 2020-07-25T16:14:12.170Z
Answers.UserDisplayName: respectful
Body: <p>I have been training my CNN for a bit now and, while both the training loss and the training error curves are going down during training, both my validation loss and my validation error curves are kind of zig-zagging and oscillating along the epochs. What does this represent?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-14T10:30:20.530Z
Comments.Text: This is an old question, but if it would be great if you could provide a plot of the training and validation loss and error. Moreover, you should provide more context, such as provide the architecture of your CNN. If you're using TF, you could do `plot_model(my_model)`.
Comments.UserDisplayName: nbro
Tags: <machine-learning><deep-learning><convolutional-neural-networks>
Title: What does an oscillating validation error curve represent?
UserDisplayName: lr99
ThreadText: # What does an oscillating validation error curve represent?
  
  (Asked by: username_0 on 2019-09-01)
  
  I have been training my CNN for a bit now and, while both the training loss and the training error curves are going down during training, both my validation loss and my validation error curves are kind of zig-zagging and oscillating along the epochs. What does this represent?
  
  **Tags:** machine-learning, deep-learning, convolutional-neural-networks
  
  > **Comments:**
  > **username_1**: This is an old question, but if it would be great if you could provide a plot of the training and validation loss and error. Moreover, you should provide more context, such as provide the architecture of your CNN. If you're using TF, you could do `plot_model(my_model)`.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-09-01. Score: 3
  
  Without more information the best diagnostic is:
  
  *You potentially have a bug in your code.*
  
  **Justification**
  
  Neural network systems are capable of failing silently. The system can still appear to "learn" even in the presence of said bug - a while back I had a very similar issue with a toy CNN project. My network could get 99% accuracy on the training set but always achieved 8-13% on the validation set. This looked like overfit but none of the methods solved the issue. Finally, I found that I wasn't feeding the data correctly into the network during train time but I **was** feeding the data correctly during validation time.
  
  **Conclusion**
  
  Provide the following for better diagnostic:
  
  * Is this a custom CNN you hardcoded?
  * are you using python?
  * what's the objective function?
  * can you provide the loss curve for the training set?
  * show us some code too (this will be very helpful for working toward a solution)
  
  I hope this helps, and best of luck!
  
  
  ---
  
  