<p>You are referring to catastrophic forgetting which could be an issue in any neural net. More specifically for DQN refer to this <a href="https://www.pnas.org/content/114/13/3521" rel="noreferrer">article</a>.</p>

CC BY-SA 4.0
2020-07-27T12:23:43.707Z
I also have a question, the issue is our replay memory size can’t be too large because of performance issues, too small and it’s irrelevant, so if we have a large space state with multiple tasks, even replay memory wouldn’t be able to help with the catastrophic forgetting, so what can be the solution?
Chukwudi
CC BY-SA 4.0
2020-07-27T13:45:55.803Z
@Chukwudi I'm not really sure, sorry.
pedrum
CC BY-SA 4.0
2020-07-27T12:13:32.620Z
2020-07-27T12:13:32.620Z
pedrum
<p>Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution?</p>
<p>For example:</p>
<ul>
<li><p>I use level 1 experiences, my model <span class="math-container">$p$</span> is fitted to learn how to play that level.</p>
</li>
<li><p>I go to level 2, my weights are updated and fitted to play level 2 meaning I don't know how to play level 1 again.</p>
</li>
</ul>

CC BY-SA 4.0
2020-07-27T13:17:08.527Z
Please, do not ask the same question in different posts only because you have not yet received an answer. You already asked about ER here: https://ai.stackexchange.com/q/22694/2444. I am sure someone will answer that question.
nbro
CC BY-SA 4.0
2020-07-28T13:10:07.943Z
This question is very related to https://ai.stackexchange.com/q/13289/2444, although I wouldn't say it's a duplicate because yours is specific to DQN.
nbro
<reinforcement-learning><q-learning><dqn><experience-replay><catastrophic-forgetting>
Why do DQNs tend to forget?
Chukwudi
# Why do DQNs tend to forget?

(Asked by: username_0 on 2020-07-27)

Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution?

For example:

* I use level 1 experiences, my model $p$ is fitted to learn how to play that level.
* I go to level 2, my weights are updated and fitted to play level 2 meaning I don't know how to play level 1 again.

**Tags:** reinforcement-learning, q-learning, dqn, experience-replay, catastrophic-forgetting

> **Comments:**
> **username_1**: Please, do not ask the same question in different posts only because you have not yet received an answer. You already asked about ER here: https://ai.stackexchange.com/q/22694/2444. I am sure someone will answer that question.  
> **username_1**: This question is very related to https://ai.stackexchange.com/q/13289/2444, although I wouldn't say it's a duplicate because yours is specific to DQN.  

---

## 1 answers

### Answer by username_2 on 2020-07-27. Score: 5 | Accepted answer

You are referring to catastrophic forgetting which could be an issue in any neural net. More specifically for DQN refer to this [article](https://www.pnas.org/content/114/13/3521).

> **Comments:**
> **username_0**: I also have a question, the issue is our replay memory size can’t be too large because of performance issues, too small and it’s irrelevant, so if we have a large space state with multiple tasks, even replay memory wouldn’t be able to help with the catastrophic forgetting, so what can be the solution?  
> **username_2**: @username_0 I'm not really sure, sorry.  

---

