Answers.Body: <p>There are several common deep reinforcement algorithms and models apart from <a href="https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf" rel="nofollow noreferrer">deep Q networks (or deep Q learning)</a>. I will list some of them below (along with a link to the paper that introduced them), but note that some of these may not be state-of-the-art (at least, not anymore, and it's likely that all of these will be replaced in the future).</p>
  <ul>
  <li><a href="https://arxiv.org/pdf/1509.06461.pdf" rel="nofollow noreferrer">Double DQN (DDQN)</a> (2015)</li>
  <li><a href="https://arxiv.org/pdf/1511.06581.pdf" rel="nofollow noreferrer">Duelling DQN</a> (2015)</li>
  <li><a href="http://proceedings.mlr.press/v37/schulman15.pdf" rel="nofollow noreferrer">Trust Region Policy Optimization (TRPO)</a> (2015)</li>
  <li><a href="https://arxiv.org/pdf/1509.02971.pdf" rel="nofollow noreferrer">Deep Deterministic Policy Gradient (DDPG)</a> (2016)</li>
  <li><a href="http://proceedings.mlr.press/v48/mniha16.pdf" rel="nofollow noreferrer">Asynchronous Advantage Actor-Critic (A3C)</a> (2016)</li>
  <li><a href="https://papers.nips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Paper.pdf" rel="nofollow noreferrer">Hindsight Experience Replay (HER)</a> (2017)</li>
  <li><a href="https://arxiv.org/pdf/1707.06347.pdf" rel="nofollow noreferrer">Proximal policy optimization (PPO)</a> (2017)</li>
  <li><a href="http://proceedings.mlr.press/v80/fujimoto18a/fujimoto18a.pdf" rel="nofollow noreferrer">Twin Delayed Deep Deterministic policy gradient algorithm (TD3)</a> (2018)</li>
  <li><a href="http://proceedings.mlr.press/v80/haarnoja18b/haarnoja18b.pdf" rel="nofollow noreferrer">Soft Actor-Critic (SAC)</a> (2018)</li>
  </ul>
  <p>For an exhaustive overview of deep RL algorithms and models, maybe take a look at this pre-print <a href="https://arxiv.org/pdf/1810.06339.pdf" rel="nofollow noreferrer">Deep Reinforcement Learning</a> (2018) by Yuxi Li.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-11T14:59:22.537Z
Answers.LastActivityDate: 2020-11-20T18:57:52.753Z
Answers.LastEditDate: 2020-11-20T18:57:52.753Z
Answers.UserDisplayName: nbro
Body: <p>Recently, I have completed Atari Breakout (<a href="https://arxiv.org/pdf/1312.5602.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1312.5602.pdf</a>) with DQN. </p>
  
  <p>Similar to DQN, what are the most common deep reinforcement learning algorithms and models in 2020? It seems that DQN is outdated and policy gradients are preferred.</p>
  
Tags: <reinforcement-learning><reference-request><deep-rl>
Title: What are the most common deep reinforcement learning algorithms and models apart from DQN?
UserDisplayName: jgauth
ThreadText: # What are the most common deep reinforcement learning algorithms and models apart from DQN?
  
  (Asked by: username_0 on 2020-04-10)
  
  Recently, I have completed Atari Breakout (<https://arxiv.org/pdf/1312.5602.pdf>) with DQN.
  
  Similar to DQN, what are the most common deep reinforcement learning algorithms and models in 2020? It seems that DQN is outdated and policy gradients are preferred.
  
  **Tags:** reinforcement-learning, reference-request, deep-rl
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-11. Score: 4 | Accepted answer
  
  There are several common deep reinforcement algorithms and models apart from [deep Q networks (or deep Q learning)](https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf). I will list some of them below (along with a link to the paper that introduced them), but note that some of these may not be state-of-the-art (at least, not anymore, and it's likely that all of these will be replaced in the future).
  
  * [Double DQN (DDQN)](https://arxiv.org/pdf/1509.06461.pdf) (2015)
  * [Duelling DQN](https://arxiv.org/pdf/1511.06581.pdf) (2015)
  * [Trust Region Policy Optimization (TRPO)](http://proceedings.mlr.press/v37/schulman15.pdf) (2015)
  * [Deep Deterministic Policy Gradient (DDPG)](https://arxiv.org/pdf/1509.02971.pdf) (2016)
  * [Asynchronous Advantage Actor-Critic (A3C)](http://proceedings.mlr.press/v48/mniha16.pdf) (2016)
  * [Hindsight Experience Replay (HER)](https://papers.nips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Paper.pdf) (2017)
  * [Proximal policy optimization (PPO)](https://arxiv.org/pdf/1707.06347.pdf) (2017)
  * [Twin Delayed Deep Deterministic policy gradient algorithm (TD3)](http://proceedings.mlr.press/v80/fujimoto18a/fujimoto18a.pdf) (2018)
  * [Soft Actor-Critic (SAC)](http://proceedings.mlr.press/v80/haarnoja18b/haarnoja18b.pdf) (2018)
  
  For an exhaustive overview of deep RL algorithms and models, maybe take a look at this pre-print [Deep Reinforcement Learning](https://arxiv.org/pdf/1810.06339.pdf) (2018) by Yuxi Li.
  
  
  ---
  
  