Answers.Body: <p>I have not used fastai library but this also happens on tensorboard when you have more than one training being recorded on the same plot.</p>
  <p>Looking at the picture, I think this is a very special type of graph because for a single LR value you have 2 loss values associated. Put in other words, you have the same LR value for different loss values. My guess it that some time-dependent issue is messing things here.</p>
  <p>Another intuition that may help to solve the issue is representing data differently. If you rotate the graph 90º left-wise you could see how the LR is evolving with different loss values. LR should be decreasing along with loss value, but in this case it is not like that either. So review how you are setting the LR too!</p>
  <p>My list of to-check would be:</p>
  <ul>
  <li>Check how you update LR according to the loss value</li>
  <li>Check the LR recorder instance is only being used by one training at a time</li>
  <li>Try plotting the value of LR (y) against your loss values (x)</li>
  <li>Try plotting the value of LR (y) against the current epoch number (x)</li>
  </ul>
  <p>Hope it helps</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-17T20:55:57.780Z
Answers.Text: no of epochs I ran is 1, and I split the running rates between (1e-5,1e-4) during training
Answers.UserDisplayName: MonkeyDLuffy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-17T17:44:28.310Z
Answers.LastActivityDate: 2020-07-17T17:44:28.310Z
Answers.UserDisplayName: JVGD
Body: <p>I'm working on recognizing the numbers 3 and 7 using the MNIST data set. I'm using <code>cnn_learner()</code> function from <a href="https://docs.fast.ai/" rel="nofollow noreferrer">fastai library</a>.</p>
  <p>When I plotted the learning rate, the curve started going backward after a certain value on X-axis. Can someone please explain what does it signify?</p>
  <p><a href="https://i.sstatic.net/SxIdz.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SxIdz.png" alt="learning rate graph" /></a></p>
  
Tags: <neural-networks><deep-learning><convolutional-neural-networks><learning-rate>
Title: Why would the learning rate curve go backwards?
UserDisplayName: MonkeyDLuffy
ThreadText: # Why would the learning rate curve go backwards?
  
  (Asked by: username_0 on 2020-07-17)
  
  I'm working on recognizing the numbers 3 and 7 using the MNIST data set. I'm using `cnn_learner()` function from [fastai library](https://docs.fast.ai/).
  
  When I plotted the learning rate, the curve started going backward after a certain value on X-axis. Can someone please explain what does it signify?
  
  [![learning rate graph](https://i.sstatic.net/SxIdz.png)](https://i.sstatic.net/SxIdz.png)
  
  **Tags:** neural-networks, deep-learning, convolutional-neural-networks, learning-rate
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-17. Score: 1 | Accepted answer
  
  I have not used fastai library but this also happens on tensorboard when you have more than one training being recorded on the same plot.
  
  Looking at the picture, I think this is a very special type of graph because for a single LR value you have 2 loss values associated. Put in other words, you have the same LR value for different loss values. My guess it that some time-dependent issue is messing things here.
  
  Another intuition that may help to solve the issue is representing data differently. If you rotate the graph 90º left-wise you could see how the LR is evolving with different loss values. LR should be decreasing along with loss value, but in this case it is not like that either. So review how you are setting the LR too!
  
  My list of to-check would be:
  
  * Check how you update LR according to the loss value
  * Check the LR recorder instance is only being used by one training at a time
  * Try plotting the value of LR (y) against your loss values (x)
  * Try plotting the value of LR (y) against the current epoch number (x)
  
  Hope it helps
  
  > **Comments:**
  > **username_0**: no of epochs I ran is 1, and I split the running rates between (1e-5,1e-4) during training  
  
  ---
  
  