Body: <p>I've found online this interesting algorithm:
  <a href="https://i.sstatic.net/z9f73.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/z9f73.png" alt="r"></a></p>
  
  <p>From what I understand reading this algorithm, I can't figure out why I should "perform the opposite action" and consequently storing that second experience as then it is never used to update or doing anything else. Is this algorithm incorrect?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-28T08:50:36.043Z
Comments.Text: It's used, they store both observations in $M$ and then they sample from it, just because they use notation $e_i = (o_i,a_i,r_i,o_{i+1})$ it doesn't mean they sample only observations from $\mathcal E$.
Comments.UserDisplayName: Brale
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-03T07:42:46.627Z
Comments.Text: @Brale just another question on this algorithm: when computing $y_i$ they use the argmax operator but w.r.t the main DNN. I've seen other pseudo algorithm which include the target network that compute the action w.r.t the target netowrk. What is the difference?
Comments.UserDisplayName: unter_983
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-03T12:46:06.570Z
Comments.Text: This is some variation of double Q learning, look up Deep Double Q Learning
Comments.UserDisplayName: Brale
Tags: <reinforcement-learning><dqn><deep-neural-networks>
Title: Understanding the role of the target network in this DQN algorithm
UserDisplayName: unter_983
ThreadText: # Understanding the role of the target network in this DQN algorithm
  
  (Asked by: username_0 on 2020-05-28)
  
  I've found online this interesting algorithm:
  [![r](https://i.sstatic.net/z9f73.png)](https://i.sstatic.net/z9f73.png)
  
  From what I understand reading this algorithm, I can't figure out why I should "perform the opposite action" and consequently storing that second experience as then it is never used to update or doing anything else. Is this algorithm incorrect?
  
  **Tags:** reinforcement-learning, dqn, deep-neural-networks
  
  > **Comments:**
  > **username_1**: It's used, they store both observations in $M$ and then they sample from it, just because they use notation $e_i = (o_i,a_i,r_i,o_{i+1})$ it doesn't mean they sample only observations from $\mathcal E$.  
  > **username_0**: @username_1 just another question on this algorithm: when computing $y_i$ they use the argmax operator but w.r.t the main DNN. I've seen other pseudo algorithm which include the target network that compute the action w.r.t the target netowrk. What is the difference?  
  > **username_1**: This is some variation of double Q learning, look up Deep Double Q Learning  
  
  ---
  
  ## 0 answers
  
  