Answers.Body: <p>For the programming part I suggest this YouTube <a href="https://www.youtube.com/channel/UC58v9cLitc8VaCjrcKyAbrw/featured" rel="noreferrer">channel</a> by Phil Tabor (he also has a website: <a href="https://www.neuralnet.ai" rel="noreferrer">neuralnet.ai</a>. I found his videos really useful while I was attending reinforcement learning classes at the uni. He covers basic algorithms like value iteration and policy iteration and also more advanced like deep q learning, covering all main python libraries (Keras, tensorflow, pytorch). Hope it will help you as well! </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-26T14:40:58.663Z
Answers.LastActivityDate: 2020-03-26T14:40:58.663Z
Answers.UserDisplayName: Edoardo Guerriero
Answers.Body: <p>Let me first say that deep RL is just the combination of RL with deep learning. So, if you study RL and deep learning, then studying deep RL should be straightforward. For this reason, this answer will point the reader to potentially useful courses on RL (also because there aren't many free courses completely dedicated to deep RL), which have at least one lesson on deep RL or function approximation. I have only followed the course by Isbell and Littman and partially the course by David Silver, so I can't ensure you that the other courses are good, but I found these two useful, although not perfect.</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>Title</th>
  <th>Instructor(s)</th>
  <th>Focus on deep RL?</th>
  <th>Topics</th>
  <th>Free</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td><a href="https://classroom.udacity.com/courses/ud600/" rel="nofollow noreferrer">Reinforcement Learning</a></td>
  <td>Charles Isbell, Michael Littman</td>
  <td>No</td>
  <td>TD learning, convergence, function approximation, POMDP, options, game theory</td>
  <td>Yes</td>
  </tr>
  <tr>
  <td><a href="https://www.youtube.com/playlist?list=PLqYmG7hTraZBiG_XpjnPrSNw-1XQaM_gB" rel="nofollow noreferrer">Introduction to Reinforcement Learning with David Silver</a></td>
  <td>David Silver</td>
  <td>No</td>
  <td>MDPs, planning, dynamic programming, model-free prediction and control, function approximation, policy gradients, exploration and exploitation</td>
  <td>Yes</td>
  </tr>
  <tr>
  <td><a href="https://www.youtube.com/watch?v=FgzM3zpZ55o&amp;list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u" rel="nofollow noreferrer">CS234: Reinforcement Learning Winter 2020</a></td>
  <td>Emma Brunskill</td>
  <td>No</td>
  <td>See the <a href="http://web.stanford.edu/class/cs234/schedule.html" rel="nofollow noreferrer">course schedule</a>; <a href="https://www.youtube.com/watch?v=gOV8-bC1_KU&amp;list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&amp;index=6" rel="nofollow noreferrer">lesson 6</a> is about DRL</td>
  <td>Yes</td>
  </tr>
  <tr>
  <td><a href="https://nptel.ac.in/courses/106/106/106106143" rel="nofollow noreferrer">Reinforcement Learning</a></td>
  <td>NPTEL</td>
  <td>No</td>
  <td>Bandits, MDPs, policy gradients, dynamic programming, TD learning, function approximation, hierarchical RL, POMDP</td>
  <td>Yes</td>
  </tr>
  <tr>
  <td><a href="https://www.youtube.com/watch?v=P9XezMuPfLE&amp;list=PL-9x0_FO_lglnlYextpvu39E7vWzHhtNO&amp;index=1&amp;ab_channel=MachineLearningwithPhil" rel="nofollow noreferrer">Reinforcement Learning in the Open AI Gym</a></td>
  <td>Phil Tabor</td>
  <td>?</td>
  <td>SARSA, double Q-learning, Monte Carlo methods, Q-learning</td>
  <td>Yes</td>
  </tr>
  <tr>
  <td><a href="https://www.youtube.com/watch?v=iOh7QUZGyiU&amp;list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs" rel="nofollow noreferrer">Advanced Deep Learning &amp; Reinforcement Learning</a></td>
  <td>DeepMind</td>
  <td>No</td>
  <td><a href="https://www.youtube.com/watch?v=L6xaQ501jEs&amp;list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs&amp;index=14" rel="nofollow noreferrer">video 14</a> discusses DRL topics</td>
  <td>Yes</td>
  </tr>
  <tr>
  <td><a href="https://www.udemy.com/course/deep-reinforcement-learning-in-python/" rel="nofollow noreferrer">Advanced AI: Deep Reinforcement Learning in Python</a></td>
  <td>Udemy</td>
  <td>Yes, it seems</td>
  <td>?</td>
  <td>No</td>
  </tr>
  <tr>
  <td><a href="https://www.udemy.com/course/machine-learning-beginner-reinforcement-learning-in-python/" rel="nofollow noreferrer">Machine Learning: Beginner Reinforcement Learning in Python</a></td>
  <td>Udemy</td>
  <td>?</td>
  <td>?</td>
  <td>No</td>
  </tr>
  <tr>
  <td><a href="https://www.udemy.com/course/deep-reinforcement-learning/" rel="nofollow noreferrer">Deep Reinforcement Learning 2.0</a></td>
  <td>Udemy</td>
  <td>?</td>
  <td>?</td>
  <td>No</td>
  </tr>
  <tr>
  <td><a href="https://www.udemy.com/course/deep-q-learning-from-paper-to-code/?couponCode=DQN-JUN-3-2020" rel="nofollow noreferrer">Modern Reinforcement Learning: Deep Q Learning in PyTorch</a></td>
  <td>Phil Tabor (Udemy)</td>
  <td>?</td>
  <td>?</td>
  <td>No</td>
  </tr>
  <tr>
  <td><a href="https://www.udemy.com/course/actor-critic-methods-from-paper-to-code-with-pytorch/?couponCode=AC-JUN-3-2020" rel="nofollow noreferrer">Modern Reinforcement Learning: Actor-Critic Methods</a></td>
  <td>Phil Tabor (Udemy)</td>
  <td>?</td>
  <td>?</td>
  <td>No</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>In any case, if you are familiar with RL and deep learning topics, I encourage you to directly read the DQN papers (both by DeepMind folks)</p>
  <ul>
  <li><a href="https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf" rel="nofollow noreferrer">Playing Atari with Deep Reinforcement Learning</a> (2013)</li>
  <li><a href="https://www.nature.com/articles/nature14236" rel="nofollow noreferrer">Human-level control through deep reinforcement learning</a> (2015)</li>
  </ul>
  <p>Of course, deep RL isn't just DQN, but these are two very important papers that you should read. Other key papers on deep RL can be found <a href="https://spinningup.openai.com/en/latest/spinningup/keypapers.html" rel="nofollow noreferrer">here</a>.</p>
  <p>Note that, depending on your experience with and knowledge of RL and DL, you may require a few iterations to fully understand these papers, but this applies every time you need to <a href="https://ai.stackexchange.com/q/22269/2444">read a research paper</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-28T01:44:21.203Z
Answers.LastActivityDate: 2022-01-14T19:35:09.167Z
Answers.LastEditDate: 2022-01-14T19:35:09.167Z
Answers.UserDisplayName: nbro
Body: <p>What are some (good) online courses for deep reinforcement learning?</p>
  
  <p>I would like the course to be both programming and theoretical. I really liked <a href="https://www.youtube.com/playlist?list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9-" rel="noreferrer">David Silver's course</a>, but the course dates from 2015. It doesn't really teach deep Q-learning at this time.</p>
  
Tags: <reinforcement-learning><q-learning><dqn><deep-rl><resource-request>
Title: What are some online courses for deep reinforcement learning?
UserDisplayName: J.Doe
ThreadText: # What are some online courses for deep reinforcement learning?
  
  (Asked by: username_0 on 2020-03-25)
  
  What are some (good) online courses for deep reinforcement learning?
  
  I would like the course to be both programming and theoretical. I really liked [David Silver's course](https://www.youtube.com/playlist?list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9-), but the course dates from 2015. It doesn't really teach deep Q-learning at this time.
  
  **Tags:** reinforcement-learning, q-learning, dqn, deep-rl, resource-request
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-03-28. Score: 7 | Accepted answer
  
  Let me first say that deep RL is just the combination of RL with deep learning. So, if you study RL and deep learning, then studying deep RL should be straightforward. For this reason, this answer will point the reader to potentially useful courses on RL (also because there aren't many free courses completely dedicated to deep RL), which have at least one lesson on deep RL or function approximation. I have only followed the course by Isbell and Littman and partially the course by David Silver, so I can't ensure you that the other courses are good, but I found these two useful, although not perfect.
  
  | Title | Instructor(s) | Focus on deep RL? | Topics | Free |
  | --- | --- | --- | --- | --- |
  | [Reinforcement Learning](https://classroom.udacity.com/courses/ud600/) | Charles Isbell, Michael Littman | No | TD learning, convergence, function approximation, POMDP, options, game theory | Yes |
  | [Introduction to Reinforcement Learning with David Silver](https://www.youtube.com/playlist?list=PLqYmG7hTraZBiG_XpjnPrSNw-1XQaM_gB) | David Silver | No | MDPs, planning, dynamic programming, model-free prediction and control, function approximation, policy gradients, exploration and exploitation | Yes |
  | [CS234: Reinforcement Learning Winter 2020](https://www.youtube.com/watch?v=FgzM3zpZ55o&list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u) | Emma Brunskill | No | See the [course schedule](http://web.stanford.edu/class/cs234/schedule.html); [lesson 6](https://www.youtube.com/watch?v=gOV8-bC1_KU&list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&index=6) is about DRL | Yes |
  | [Reinforcement Learning](https://nptel.ac.in/courses/106/106/106106143) | NPTEL | No | Bandits, MDPs, policy gradients, dynamic programming, TD learning, function approximation, hierarchical RL, POMDP | Yes |
  | [Reinforcement Learning in the Open AI Gym](https://www.youtube.com/watch?v=P9XezMuPfLE&list=PL-9x0_FO_lglnlYextpvu39E7vWzHhtNO&index=1&ab_channel=MachineLearningwithPhil) | Phil Tabor | ? | SARSA, double Q-learning, Monte Carlo methods, Q-learning | Yes |
  | [Advanced Deep Learning & Reinforcement Learning](https://www.youtube.com/watch?v=iOh7QUZGyiU&list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs) | DeepMind | No | [video 14](https://www.youtube.com/watch?v=L6xaQ501jEs&list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs&index=14) discusses DRL topics | Yes |
  | [Advanced AI: Deep Reinforcement Learning in Python](https://www.udemy.com/course/deep-reinforcement-learning-in-python/) | Udemy | Yes, it seems | ? | No |
  | [Machine Learning: Beginner Reinforcement Learning in Python](https://www.udemy.com/course/machine-learning-beginner-reinforcement-learning-in-python/) | Udemy | ? | ? | No |
  | [Deep Reinforcement Learning 2.0](https://www.udemy.com/course/deep-reinforcement-learning/) | Udemy | ? | ? | No |
  | [Modern Reinforcement Learning: Deep Q Learning in PyTorch](https://www.udemy.com/course/deep-q-learning-from-paper-to-code/?couponCode=DQN-JUN-3-2020) | Phil Tabor (Udemy) | ? | ? | No |
  | [Modern Reinforcement Learning: Actor-Critic Methods](https://www.udemy.com/course/actor-critic-methods-from-paper-to-code-with-pytorch/?couponCode=AC-JUN-3-2020) | Phil Tabor (Udemy) | ? | ? | No |
  
  In any case, if you are familiar with RL and deep learning topics, I encourage you to directly read the DQN papers (both by DeepMind folks)
  
  * [Playing Atari with Deep Reinforcement Learning](https://www.cs.toronto.edu/%7Evmnih/docs/dqn.pdf) (2013)
  * [Human-level control through deep reinforcement learning](https://www.nature.com/articles/nature14236) (2015)
  
  Of course, deep RL isn't just DQN, but these are two very important papers that you should read. Other key papers on deep RL can be found [here](https://spinningup.openai.com/en/latest/spinningup/keypapers.html).
  
  Note that, depending on your experience with and knowledge of RL and DL, you may require a few iterations to fully understand these papers, but this applies every time you need to [read a research paper](https://ai.stackexchange.com/q/22269/2444).
  
  
  ---
  
  ### Answer by username_1 on 2020-03-26. Score: 5
  
  For the programming part I suggest this YouTube [channel](https://www.youtube.com/channel/UC58v9cLitc8VaCjrcKyAbrw/featured) by Phil Tabor (he also has a website: [neuralnet.ai](https://www.neuralnet.ai). I found his videos really useful while I was attending reinforcement learning classes at the uni. He covers basic algorithms like value iteration and policy iteration and also more advanced like deep q learning, covering all main python libraries (Keras, tensorflow, pytorch). Hope it will help you as well!
  
  
  ---
  
  