Answers.Body: <p>I think it says something about the training progress, while another approach you can make sure is to look at the gradient norm. Sometimes, the training loss is really noisy while the gradient norm is much more clear.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-12T00:51:01.813Z
Answers.Text: u mean the gradient of the weights of the model ? what would the gradient norm say about the training ?
Answers.UserDisplayName: calveeen
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-11T00:55:43.687Z
Answers.Text: well smaller norm would indicate that training is about to converge i guess. At a local minima, all the dimensions of a gradient would be 0.
Answers.UserDisplayName: SpiderRico
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-11T19:43:56.727Z
Answers.LastActivityDate: 2020-05-11T19:43:56.727Z
Answers.UserDisplayName: DongDong Chen
Body: <p>During the training of DQN, I noticed that the model with prioritized experience replay (PER) had a smaller loss in general compared to a DQN without PER. The mean squared loss was an order of magnitude <span class="math-container">$10^{-5}$</span> for the DQN with PER, whereas the mean squared loss was an order of magnitude <span class="math-container">$10^{-2}$</span>. </p>
  
  <p>Do the smaller training errors have any effect on executing the final policy learned by the DQN?</p>
  
Tags: <reinforcement-learning><dqn><loss>
Title: Do smaller loss values during DQN training produce better policies?
UserDisplayName: calveeen
ThreadText: # Do smaller loss values during DQN training produce better policies?
  
  (Asked by: username_0 on 2020-05-11)
  
  During the training of DQN, I noticed that the model with prioritized experience replay (PER) had a smaller loss in general compared to a DQN without PER. The mean squared loss was an order of magnitude $10^{-5}$ for the DQN with PER, whereas the mean squared loss was an order of magnitude $10^{-2}$.
  
  Do the smaller training errors have any effect on executing the final policy learned by the DQN?
  
  **Tags:** reinforcement-learning, dqn, loss
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-11. Score: 1
  
  I think it says something about the training progress, while another approach you can make sure is to look at the gradient norm. Sometimes, the training loss is really noisy while the gradient norm is much more clear.
  
  > **Comments:**
  > **username_0**: u mean the gradient of the weights of the model ? what would the gradient norm say about the training ?  
  > **username_2**: well smaller norm would indicate that training is about to converge i guess. At a local minima, all the dimensions of a gradient would be 0.  
  
  ---
  
  