abstract
deep reinforcement learning has emerged as a powerful tool
for a variety of learning tasks, however deep nets typically exhibit forgetting when learning multiple tasks in sequence. to
mitigate forgetting, we propose an experience replay process
that augments the standard fifo buffer and selectively stores
experiences in a long-term memory. we explore four strategies for selecting which experiences will be stored: favoring
surprise, favoring reward, matching the global training distribution, and maximizing coverage of the state space. we show
that distribution matching successfully prevents catastrophic
forgetting, and is consistently the best approach on all domains tested. while distribution matching has better and more
consistent performance, we identify one case in which coverage maximization is beneficial - when tasks that receive less
trained are more important. overall, our results show that selective experience replay, when suitable selection algorithms
are employed, can prevent catastrophic forgetting.