Body: <p>Whenever you are doing reinforcement learning you periodically update the target network based on the weights of the Q network. While I do understand this helps create a stable target I do not understand how it helps the Q networks accuracy. If I am updating the Q network much more frequently than I am the target network then wont the Q network converge to and estimate of the target network that is inaccurate since the target network is being updated much slower ?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-02T13:50:27.777Z
Comments.Text: I think this https://ai.stackexchange.com/a/13459/69468 already answers your question
Comments.UserDisplayName: Luca Anzalone
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-02T16:59:07.557Z
Comments.Text: oh wow perfect this is exactly what I needed
Comments.UserDisplayName: Stef
Tags: <reinforcement-learning><deep-learning><deep-rl><dqn><deep-neural-networks>
Title: In DQN how does the Q network not converge to the incorrect target?
UserDisplayName: Stef
ThreadText: # In DQN how does the Q network not converge to the incorrect target?
  
  (Asked by: username_0 on 2023-07-01)
  
  Whenever you are doing reinforcement learning you periodically update the target network based on the weights of the Q network. While I do understand this helps create a stable target I do not understand how it helps the Q networks accuracy. If I am updating the Q network much more frequently than I am the target network then wont the Q network converge to and estimate of the target network that is inaccurate since the target network is being updated much slower ?
  
  **Tags:** reinforcement-learning, deep-learning, deep-rl, dqn, deep-neural-networks
  
  > **Comments:**
  > **username_1**: I think this https://ai.stackexchange.com/a/13459/69468 already answers your question  
  > **username_0**: oh wow perfect this is exactly what I needed  
  
  ---
  
  ## 0 answers
  
  