Answers.Body: <p>In addition to the ones you mentioned, I would add <a href="https://sites.ualberta.ca/%7Eszepesva/rlbook.html" rel="nofollow noreferrer">Algorithms of Reinforcement Learning</a> by Csaba Szepesvári. There is a number of professors who use it as a reference in their RL teaching materials (for example <a href="https://cs.uwaterloo.ca/%7Eppoupart/teaching/cs885-spring20/textbook.html" rel="nofollow noreferrer">this one</a>).</p>
  <p>It generally follows the same outline as Sutton &amp; Barto's book (except the part on bandits, it is included in the Chapter on <em>Control</em>). In fact, it may be considered as a condensed version of Sutton &amp; Barto (about 100 pages). In addition, it's freely available online.</p>
  <p>I like the author's justification as to why he wrote this book, so I'm just going to quote it:</p>
  <blockquote>
  <p>Why did I write this book? Good question! There exist a good number of
  really great books on Reinforcement Learning. So why a new book? I had
  selfish reasons: I wanted a <strong>short</strong> book, which nevertheless contained
  the major ideas underlying <strong>state-of-the-art RL algorithms</strong> (back in
  2010), a discussion of their relative <strong>strengths</strong> and <strong>weaknesses</strong>, with
  hints on what is known (and not known, but would be good to know)
  about these algorithms.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-19T13:31:31.670Z
Answers.LastActivityDate: 2020-06-19T13:31:31.670Z
Answers.UserDisplayName: user5093249
Answers.Body: <p>The (draft) book <a href="https://rltheorybook.github.io/rltheorybook_AJKS.pdf" rel="nofollow noreferrer">Reinforcement Learning: Theory and Algorithms</a>, by <a href="https://sham.seas.harvard.edu/" rel="nofollow noreferrer">Sham M. Kakade</a> (who published a <a href="https://proceedings.neurips.cc/paper/2001/file/4b86abe48d358ecf194c56c69108433e-Paper.pdf" rel="nofollow noreferrer">natural policy gradient</a> algorithm and other important research) and others, introduces RL in a mathematical/formal way. It seems to me that this is a reliable book, but a bit advanced for &quot;regular people&quot;. Yes, I know the question was about introductory books on RL, but this may be suitable for people that have a solid knowledge of math and would like a hardcore intro to RL. For example, the book starts with a non-trivial (in my view) proof that there exists an optimal stationary and deterministic policy for an MDP.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-01T22:47:52.140Z
Answers.LastActivityDate: 2023-02-01T22:47:52.140Z
Answers.UserDisplayName: nbro
Answers.Body: <p><a href="https://www.pearson.com/us/higher-education/program/Graesser-Foundations-of-Deep-Reinforcement-Learning-Theory-and-Practice-in-Python/PGM2027228.html" rel="nofollow noreferrer">Foundations of Deep Reinforcement Learning: Theory and Practice in Python (Addison-Wesley Data &amp; Analytics Series) 1st Edition</a></p>
  <p>This book does not give a detailed background information on Markov Decision Processes, different Bellman equations and relationships between the value function and action-value function, etc. It focuses on Deep Reinforcement Learning and goes straight to Policy and Value - based algorithms using neural networks. It might be good for someone trying to quickly understand what Deep RL algorithms are out there and apply them.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-14T10:42:11.753Z
Answers.LastActivityDate: 2020-08-14T10:42:11.753Z
Answers.UserDisplayName: tmaric
Body: <p>Currently, I'm only going through these two books</p>
  <ul>
  <li><p><a href="http://incompleteideas.net/book/RLbook2020.pdf" rel="nofollow noreferrer">Reinforcement Learning: An Introduction, by Sutton and Barto</a>: RL explained on an engineering level (mathematical, but readable for a non-mathematician). Elementary notions from probability and statistics are required (conditional probability, total probability theorem, total expectation theorem, and similar. The <a href="https://www.youtube.com/watch?v=1uW3qMFA9Ho&amp;list=PLUl4u3cNGP60hI9ATjSFgLZpbNJ7myAg6" rel="nofollow noreferrer">MIT RES.6-012 &quot;Introduction to Probability&quot;</a> course is a great source of information for these topics.).</p>
  </li>
  <li><p><a href="https://www.manning.com/books/grokking-deep-reinforcement-learning" rel="nofollow noreferrer">Deep Reinforcement Learning, by Miguel Morales</a>: this book introduces the main elements of reinforcement learning in a less formal way than Sutton and Barto (derivations for some equations are not given), using examples to describe the math.</p>
  </li>
  </ul>
  <p>What other introductory books to reinforcement learning do you know, and how do they approach this topic?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-19T08:28:12.603Z
Comments.Text: I expect a good answer to provide links to other books but also a brief description of the book and how they treat the subject.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><reference-request><books>
Title: What introductory books to reinforcement learning do you know, and how do they approach this topic?
UserDisplayName: tmaric
ThreadText: # What introductory books to reinforcement learning do you know, and how do they approach this topic?
  
  (Asked by: username_0 on 2020-06-19)
  
  Currently, I'm only going through these two books
  
  * [Reinforcement Learning: An Introduction, by Sutton and Barto](http://incompleteideas.net/book/RLbook2020.pdf): RL explained on an engineering level (mathematical, but readable for a non-mathematician). Elementary notions from probability and statistics are required (conditional probability, total probability theorem, total expectation theorem, and similar. The [MIT RES.6-012 "Introduction to Probability"](https://www.youtube.com/watch?v=1uW3qMFA9Ho&list=PLUl4u3cNGP60hI9ATjSFgLZpbNJ7myAg6) course is a great source of information for these topics.).
  * [Deep Reinforcement Learning, by Miguel Morales](https://www.manning.com/books/grokking-deep-reinforcement-learning): this book introduces the main elements of reinforcement learning in a less formal way than Sutton and Barto (derivations for some equations are not given), using examples to describe the math.
  
  What other introductory books to reinforcement learning do you know, and how do they approach this topic?
  
  **Tags:** reinforcement-learning, reference-request, books
  > **Comments:**
  > **username_1**: I expect a good answer to provide links to other books but also a brief description of the book and how they treat the subject.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2020-06-19. Score: 3
  
  In addition to the ones you mentioned, I would add [Algorithms of Reinforcement Learning](https://sites.ualberta.ca/%7Eszepesva/rlbook.html) by Csaba Szepesvári. There is a number of professors who use it as a reference in their RL teaching materials (for example [this one](https://cs.uwaterloo.ca/%7Eppoupart/teaching/cs885-spring20/textbook.html)).
  
  It generally follows the same outline as Sutton & Barto's book (except the part on bandits, it is included in the Chapter on *Control*). In fact, it may be considered as a condensed version of Sutton & Barto (about 100 pages). In addition, it's freely available online.
  
  I like the author's justification as to why he wrote this book, so I'm just going to quote it:
  
  > Why did I write this book? Good question! There exist a good number of
  > really great books on Reinforcement Learning. So why a new book? I had
  > selfish reasons: I wanted a **short** book, which nevertheless contained
  > the major ideas underlying **state-of-the-art RL algorithms** (back in
  > 2010), a discussion of their relative **strengths** and **weaknesses**, with
  > hints on what is known (and not known, but would be good to know)
  > about these algorithms.
  
  
  ---
  
  ### Answer by username_1 on 2023-02-01. Score: 2
  
  The (draft) book [Reinforcement Learning: Theory and Algorithms](https://rltheorybook.github.io/rltheorybook_AJKS.pdf), by [Sham M. Kakade](https://sham.seas.harvard.edu/) (who published a [natural policy gradient](https://proceedings.neurips.cc/paper/2001/file/4b86abe48d358ecf194c56c69108433e-Paper.pdf) algorithm and other important research) and others, introduces RL in a mathematical/formal way. It seems to me that this is a reliable book, but a bit advanced for "regular people". Yes, I know the question was about introductory books on RL, but this may be suitable for people that have a solid knowledge of math and would like a hardcore intro to RL. For example, the book starts with a non-trivial (in my view) proof that there exists an optimal stationary and deterministic policy for an MDP.
  
  
  ---
  
  ### Answer by username_0 on 2020-08-14. Score: 1
  
  [Foundations of Deep Reinforcement Learning: Theory and Practice in Python (Addison-Wesley Data & Analytics Series) 1st Edition](https://www.pearson.com/us/higher-education/program/Graesser-Foundations-of-Deep-Reinforcement-Learning-Theory-and-Practice-in-Python/PGM2027228.html)
  
  This book does not give a detailed background information on Markov Decision Processes, different Bellman equations and relationships between the value function and action-value function, etc. It focuses on Deep Reinforcement Learning and goes straight to Policy and Value - based algorithms using neural networks. It might be good for someone trying to quickly understand what Deep RL algorithms are out there and apply them.
  
  
  ---
  
  