Answers.Body: <p>It looks like your network is overfitting, because the training loss carries on decreasing to zero even though validation loss levels off, and then starts to increase again.</p>
  <p>I would guess that your network is essentially &quot;memorising&quot; the training examples because you're getting a near zero loss in training.</p>
  <p>You could try:</p>
  <ul>
  <li>applying some form of regularisation</li>
  <li>reducing the number of neurons or layers</li>
  <li>collecting more training data to see if that helps.</li>
  </ul>
  <p>Ideally you want to minimise validation loss, so epoch 20 was about the best point in your graph. Look into early stopping: this is an effective way to avoid overfitting too much.</p>
  <p>When the learning rate is too high, you often see the loss 'jumping around' quite rapidly, and when the learning rate is too low, the graph barely changes at all. You could try fixing the learning rate closer to what it is around epoch 10 and see if this is &quot;the sweet spot&quot;; it's mostly a game of experimenting to see which works best.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-07T12:11:05.050Z
Answers.LastActivityDate: 2021-04-07T12:11:05.050Z
Answers.UserDisplayName: htl
Body: <p>How to interpret the following learning curves?</p>
  <p>Background: The accuracy starts at 50%, because the network has a binary output (0 or 1). I chose an exponentially decreasing learning rate of the optimizer - I believe that this the reason why the network starts learning after 10 epochs or so.</p>
  <pre><code>lr_schedule = keras.optimizers.schedules.ExponentialDecay(**h_p[&quot;optimizer&quot;][&quot;adam&quot;])
  optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
  h_p[&quot;Compile&quot;][&quot;optimizer&quot;] = optimizer
  </code></pre>
  <p><a href="https://i.sstatic.net/pcXuX.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/pcXuX.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-04-07T17:13:20.010Z
Comments.Text: This is a textbook case of overfitting. Your network fits particular patterns existing only on the training data, and fails to generalize. Given your training curves, you should stop training at about 20th epoch, in which the validation loss is the lowest.
Comments.UserDisplayName: SpiderRico
Tags: <neural-networks><machine-learning><convolutional-neural-networks><training>
Title: How to interpret this learning curve of my neural network?
UserDisplayName: David Weinblumer
ThreadText: # How to interpret this learning curve of my neural network?
  
  (Asked by: username_0 on 2021-04-07)
  
  How to interpret the following learning curves?
  Background: The accuracy starts at 50%, because the network has a binary output (0 or 1). I chose an exponentially decreasing learning rate of the optimizer - I believe that this the reason why the network starts learning after 10 epochs or so.
  lr_schedule = keras.optimizers.schedules.ExponentialDecay(**h_p["optimizer"]["adam"])
  optimizer = keras.optimizers.Adam(learning_rate=lr_schedule)
  h_p["Compile"]["optimizer"] = optimizer
  
  **Tags:** neural-networks, machine-learning, convolutional-neural-networks, training
  > **Comments:**
  > **username_1**: This is a textbook case of overfitting. Your network fits particular patterns existing only on the training data, and fails to generalize. Given your training curves, you should stop training at about 20th epoch, in which the validation loss is the lowest.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-04-07. Score: 1
  
  It looks like your network is overfitting, because the training loss carries on decreasing to zero even though validation loss levels off, and then starts to increase again.
  
  I would guess that your network is essentially "memorising" the training examples because you're getting a near zero loss in training.
  
  You could try:
  
  * applying some form of regularisation
  * reducing the number of neurons or layers
  * collecting more training data to see if that helps.
  
  Ideally you want to minimise validation loss, so epoch 20 was about the best point in your graph. Look into early stopping: this is an effective way to avoid overfitting too much.
  
  When the learning rate is too high, you often see the loss 'jumping around' quite rapidly, and when the learning rate is too low, the graph barely changes at all. You could try fixing the learning rate closer to what it is around epoch 10 and see if this is "the sweet spot"; it's mostly a game of experimenting to see which works best.
  
  
  ---
  
  