<p>There are several common deep reinforcement algorithms and models apart from <a href="https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf" rel="nofollow noreferrer">deep Q networks (or deep Q learning)</a>. I will list some of them below (along with a link to the paper that introduced them), but note that some of these may not be state-of-the-art (at least, not anymore, and it's likely that all of these will be replaced in the future).</p>
<ul>
<li><a href="https://arxiv.org/pdf/1509.06461.pdf" rel="nofollow noreferrer">Double DQN (DDQN)</a> (2015)</li>
<li><a href="https://arxiv.org/pdf/1511.06581.pdf" rel="nofollow noreferrer">Duelling DQN</a> (2015)</li>
<li><a href="http://proceedings.mlr.press/v37/schulman15.pdf" rel="nofollow noreferrer">Trust Region Policy Optimization (TRPO)</a> (2015)</li>
<li><a href="https://arxiv.org/pdf/1509.02971.pdf" rel="nofollow noreferrer">Deep Deterministic Policy Gradient (DDPG)</a> (2016)</li>
<li><a href="http://proceedings.mlr.press/v48/mniha16.pdf" rel="nofollow noreferrer">Asynchronous Advantage Actor-Critic (A3C)</a> (2016)</li>
<li><a href="https://papers.nips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Paper.pdf" rel="nofollow noreferrer">Hindsight Experience Replay (HER)</a> (2017)</li>
<li><a href="https://arxiv.org/pdf/1707.06347.pdf" rel="nofollow noreferrer">Proximal policy optimization (PPO)</a> (2017)</li>
<li><a href="http://proceedings.mlr.press/v80/fujimoto18a/fujimoto18a.pdf" rel="nofollow noreferrer">Twin Delayed Deep Deterministic policy gradient algorithm (TD3)</a> (2018)</li>
<li><a href="http://proceedings.mlr.press/v80/haarnoja18b/haarnoja18b.pdf" rel="nofollow noreferrer">Soft Actor-Critic (SAC)</a> (2018)</li>
</ul>
<p>For an exhaustive overview of deep RL algorithms and models, maybe take a look at this pre-print <a href="https://arxiv.org/pdf/1810.06339.pdf" rel="nofollow noreferrer">Deep Reinforcement Learning</a> (2018) by Yuxi Li.</p>

CC BY-SA 4.0
2020-04-11T14:59:22.537Z
2020-11-20T18:57:52.753Z
2020-11-20T18:57:52.753Z
nbro
<p>Recently, I have completed Atari Breakout (<a href="https://arxiv.org/pdf/1312.5602.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1312.5602.pdf</a>) with DQN. </p>

<p>Similar to DQN, what are the most common deep reinforcement learning algorithms and models in 2020? It seems that DQN is outdated and policy gradients are preferred.</p>

<reinforcement-learning><reference-request><deep-rl>
What are the most common deep reinforcement learning algorithms and models apart from DQN?
jgauth
# What are the most common deep reinforcement learning algorithms and models apart from DQN?

(Asked by: username_0 on 2020-04-10)

Recently, I have completed Atari Breakout (<https://arxiv.org/pdf/1312.5602.pdf>) with DQN.

Similar to DQN, what are the most common deep reinforcement learning algorithms and models in 2020? It seems that DQN is outdated and policy gradients are preferred.

**Tags:** reinforcement-learning, reference-request, deep-rl


---

## 1 answers

### Answer by username_1 on 2020-04-11. Score: 4 | Accepted answer

There are several common deep reinforcement algorithms and models apart from [deep Q networks (or deep Q learning)](https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf). I will list some of them below (along with a link to the paper that introduced them), but note that some of these may not be state-of-the-art (at least, not anymore, and it's likely that all of these will be replaced in the future).

* [Double DQN (DDQN)](https://arxiv.org/pdf/1509.06461.pdf) (2015)
* [Duelling DQN](https://arxiv.org/pdf/1511.06581.pdf) (2015)
* [Trust Region Policy Optimization (TRPO)](http://proceedings.mlr.press/v37/schulman15.pdf) (2015)
* [Deep Deterministic Policy Gradient (DDPG)](https://arxiv.org/pdf/1509.02971.pdf) (2016)
* [Asynchronous Advantage Actor-Critic (A3C)](http://proceedings.mlr.press/v48/mniha16.pdf) (2016)
* [Hindsight Experience Replay (HER)](https://papers.nips.cc/paper/2017/file/453fadbd8a1a3af50a9df4df899537b5-Paper.pdf) (2017)
* [Proximal policy optimization (PPO)](https://arxiv.org/pdf/1707.06347.pdf) (2017)
* [Twin Delayed Deep Deterministic policy gradient algorithm (TD3)](http://proceedings.mlr.press/v80/fujimoto18a/fujimoto18a.pdf) (2018)
* [Soft Actor-Critic (SAC)](http://proceedings.mlr.press/v80/haarnoja18b/haarnoja18b.pdf) (2018)

For an exhaustive overview of deep RL algorithms and models, maybe take a look at this pre-print [Deep Reinforcement Learning](https://arxiv.org/pdf/1810.06339.pdf) (2018) by Yuxi Li.


---

