Answers.Body: <p>You are referring to catastrophic forgetting which could be an issue in any neural net. More specifically for DQN refer to this <a href="https://www.pnas.org/content/114/13/3521" rel="noreferrer">article</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-27T12:23:43.707Z
Answers.Text: I also have a question, the issue is our replay memory size can’t be too large because of performance issues, too small and it’s irrelevant, so if we have a large space state with multiple tasks, even replay memory wouldn’t be able to help with the catastrophic forgetting, so what can be the solution?
Answers.UserDisplayName: Chukwudi
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-27T13:45:55.803Z
Answers.Text: @Chukwudi I'm not really sure, sorry.
Answers.UserDisplayName: pedrum
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-27T12:13:32.620Z
Answers.LastActivityDate: 2020-07-27T12:13:32.620Z
Answers.UserDisplayName: pedrum
Body: <p>Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution?</p>
  <p>For example:</p>
  <ul>
  <li><p>I use level 1 experiences, my model <span class="math-container">$p$</span> is fitted to learn how to play that level.</p>
  </li>
  <li><p>I go to level 2, my weights are updated and fitted to play level 2 meaning I don't know how to play level 1 again.</p>
  </li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-27T13:17:08.527Z
Comments.Text: Please, do not ask the same question in different posts only because you have not yet received an answer. You already asked about ER here: https://ai.stackexchange.com/q/22694/2444. I am sure someone will answer that question.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-28T13:10:07.943Z
Comments.Text: This question is very related to https://ai.stackexchange.com/q/13289/2444, although I wouldn't say it's a duplicate because yours is specific to DQN.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><q-learning><dqn><experience-replay><catastrophic-forgetting>
Title: Why do DQNs tend to forget?
UserDisplayName: Chukwudi
ThreadText: # Why do DQNs tend to forget?
  
  (Asked by: username_0 on 2020-07-27)
  
  Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution?
  
  For example:
  
  * I use level 1 experiences, my model $p$ is fitted to learn how to play that level.
  * I go to level 2, my weights are updated and fitted to play level 2 meaning I don't know how to play level 1 again.
  
  **Tags:** reinforcement-learning, q-learning, dqn, experience-replay, catastrophic-forgetting
  
  > **Comments:**
  > **username_1**: Please, do not ask the same question in different posts only because you have not yet received an answer. You already asked about ER here: https://ai.stackexchange.com/q/22694/2444. I am sure someone will answer that question.  
  > **username_1**: This question is very related to https://ai.stackexchange.com/q/13289/2444, although I wouldn't say it's a duplicate because yours is specific to DQN.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-07-27. Score: 5 | Accepted answer
  
  You are referring to catastrophic forgetting which could be an issue in any neural net. More specifically for DQN refer to this [article](https://www.pnas.org/content/114/13/3521).
  
  > **Comments:**
  > **username_0**: I also have a question, the issue is our replay memory size can’t be too large because of performance issues, too small and it’s irrelevant, so if we have a large space state with multiple tasks, even replay memory wouldn’t be able to help with the catastrophic forgetting, so what can be the solution?  
  > **username_2**: @username_0 I'm not really sure, sorry.  
  
  ---
  
  