Answers.Body: <p>If you are certain that you reached convergence then there is no point in continuing to train your agent, because of that there is also no point in discussing why is target network being updated after convergence is reached. You should simply stop training if you converged. During training we obviously need to keep updating target network to improve correctness of Q-value estimates.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-26T14:08:48.767Z
Answers.LastActivityDate: 2020-06-26T14:08:48.767Z
Answers.UserDisplayName: Brale
Body: <p>I know we keep the target network constant during training to improve stability, but why exactly are we updating the weights of our target network? In particular, if we've already reached convergence, why exactly are we updating the weights of our target network?</p>
  
Tags: <reinforcement-learning><q-learning><dqn><deep-rl>
Title: Why do we update the weights of the target network in deep Q learning?
UserDisplayName: Chukwudi
ThreadText: # Why do we update the weights of the target network in deep Q learning?
  
  (Asked by: username_0 on 2020-06-26)
  
  I know we keep the target network constant during training to improve stability, but why exactly are we updating the weights of our target network? In particular, if we've already reached convergence, why exactly are we updating the weights of our target network?
  
  **Tags:** reinforcement-learning, q-learning, dqn, deep-rl
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-26. Score: 2 | Accepted answer
  
  If you are certain that you reached convergence then there is no point in continuing to train your agent, because of that there is also no point in discussing why is target network being updated after convergence is reached. You should simply stop training if you converged. During training we obviously need to keep updating target network to improve correctness of Q-value estimates.
  
  
  ---
  
  