Answers.Body: <p>Random sampling from the replay memory helps us reach closer to the i.i.d assumption and also lets the learning algorithm train on the data multiple times compared to just once. Therefore, there is a lower chance of the network forgetting what it previously learned. However, the experience replay size is a hyperparameter and there are disadvantages of having it too low or too big. While the former disadvantage is self evident, the latter has been explained in this paper - <a href="https://arxiv.org/abs/1712.01275#" rel="nofollow noreferrer">A Deeper Look at Experience Replay</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-30T20:00:51.170Z
Answers.LastActivityDate: 2022-06-30T20:00:51.170Z
Answers.UserDisplayName: desert_ranger
Body: <p>Let's suppose that our RL agent needs to play a game with different levels. If we train our RL agent sequentially or with sequential data, our agent will learn how to play level 1, but then it will learn to play level 2 differently, because our agent learns how to play level 2 and forgets how to play level 1, since now our model is fitted using only experiences from level 2.</p>
  <p>How does an experience replay buffer change this? Can you explain this in simple terms?</p>
  
Tags: <reinforcement-learning><experience-replay>
Title: What is the advantage of using experience replay (as opposed to feeding it sequential data)?
UserDisplayName: Chukwudi
ThreadText: # What is the advantage of using experience replay (as opposed to feeding it sequential data)?
  
  (Asked by: username_0 on 2020-07-27)
  
  Let's suppose that our RL agent needs to play a game with different levels. If we train our RL agent sequentially or with sequential data, our agent will learn how to play level 1, but then it will learn to play level 2 differently, because our agent learns how to play level 2 and forgets how to play level 1, since now our model is fitted using only experiences from level 2.
  
  How does an experience replay buffer change this? Can you explain this in simple terms?
  
  **Tags:** reinforcement-learning, experience-replay
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-06-30. Score: 1
  
  Random sampling from the replay memory helps us reach closer to the i.i.d assumption and also lets the learning algorithm train on the data multiple times compared to just once. Therefore, there is a lower chance of the network forgetting what it previously learned. However, the experience replay size is a hyperparameter and there are disadvantages of having it too low or too big. While the former disadvantage is self evident, the latter has been explained in this paper - [A Deeper Look at Experience Replay](https://arxiv.org/abs/1712.01275#)
  
  
  ---
  
  