Answers.Body: <p>Decrease of loss does not essentially lead to increase of accuracy (most of the time it happens but sometime it may not happen). To know why, you can have a look at <a href="https://stats.stackexchange.com/questions/282160/how-is-it-possible-that-validation-loss-is-increasing-while-validation-accuracy">this</a> question. The network cares about decreasing the loss and it does not care about the accuracy at all. So it's no surprise to see what you presented.</p>
  <p><em>Additional note</em>: If you use <strong>batch</strong> approaches to teach your network, <em>or</em> if you choose a <strong>big step size</strong> you may also see that the loss increases sometimes but in the case of batching, the hope is that its trend is to be decreased.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-03T17:55:48.233Z
Answers.LastActivityDate: 2020-11-03T17:55:48.233Z
Answers.UserDisplayName: amin
Body: <p>I've been trying to find the optimal number of epochs that I should train my neural network (that I just implemented) for.</p>
  <p>The visualizations below show the neural network being run with a variable number of epochs. It is quite obvious that the accuracy increases with the number of epochs. However, at 75 epochs, we see a dip before the accuracy continues to rise. What is the cause of this?</p>
  <p><a href="https://i.sstatic.net/ZRIlj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ZRIlj.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-05T11:30:05.427Z
Comments.Text: You really need to provide more details about your problem, neural network and data. Is your model a simple multi-layer perceptron (i.e. not recurrent connections or convolutions)? How many layers does it have (and how many neurons per layer)? If you are using the MSE loss, I suppose that you are using a regression problem, but you're using the accuracy... So, which problem exactly? How many data points do you have? Please, edit your post to include all these details.
Comments.UserDisplayName: nbro
Tags: <neural-networks><overfitting><loss><accuracy><epochs>
Title: Why does the accuracy drop while the loss decrease, as the number of epochs increases?
UserDisplayName: eGood
ThreadText: # Why does the accuracy drop while the loss decrease, as the number of epochs increases?
  
  (Asked by: username_0 on 2020-11-03)
  
  I've been trying to find the optimal number of epochs that I should train my neural network (that I just implemented) for.
  
  The visualizations below show the neural network being run with a variable number of epochs. It is quite obvious that the accuracy increases with the number of epochs. However, at 75 epochs, we see a dip before the accuracy continues to rise. What is the cause of this?
  
  [![enter image description here](https://i.sstatic.net/ZRIlj.png)](https://i.sstatic.net/ZRIlj.png)
  
  **Tags:** neural-networks, overfitting, loss, accuracy, epochs
  
  > **Comments:**
  > **username_1**: You really need to provide more details about your problem, neural network and data. Is your model a simple multi-layer perceptron (i.e. not recurrent connections or convolutions)? How many layers does it have (and how many neurons per layer)? If you are using the MSE loss, I suppose that you are using a regression problem, but you're using the accuracy... So, which problem exactly? How many data points do you have? Please, edit your post to include all these details.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-11-03. Score: 1
  
  Decrease of loss does not essentially lead to increase of accuracy (most of the time it happens but sometime it may not happen). To know why, you can have a look at [this](https://stats.stackexchange.com/questions/282160/how-is-it-possible-that-validation-loss-is-increasing-while-validation-accuracy) question. The network cares about decreasing the loss and it does not care about the accuracy at all. So it's no surprise to see what you presented.
  
  *Additional note*: If you use **batch** approaches to teach your network, *or* if you choose a **big step size** you may also see that the loss increases sometimes but in the case of batching, the hope is that its trend is to be decreased.
  
  
  ---
  
  