<p>The more you read, the more deeply you can understand any paper, but given your stated background, reading the Monte-Carlo Tree Search chapter of Barto &amp; Sutton, plus Gerald Tesauro's <a href="https://cling.csd.uwo.ca/cs346a/extra/tdgammon.pdf" rel="nofollow noreferrer">TD-Gammon paper</a> (which is pretty accessible, and which is the basis for the <em>other</em> technique used in AlphaZero) should be enough to get a pretty good idea of what they did. </p>

CC BY-SA 4.0
2019-06-19T00:18:36.000Z
2019-06-19T20:49:32.273Z
2019-06-19T20:49:32.273Z
John Doucette
<p>My goal is to understand AlphaZero paper published by deepmind. I'm beginning my journey trying to get the basic intuition of reinforcement learning from the <a href="https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf" rel="nofollow noreferrer">book</a> by Barto and Sutton.</p>
<p>As per my background, I'm familiar with MDPs, value iteration and policy iteration.</p>
<p>I wanted to ask until what chapter of Barto and Sutton's book is one required to read in order to fully comprehend AlphaZero's paper. Monte-Carlo Tree Search is discussed in Chapter-8 of the book. Will it be enough till that? Or would I be needing more resources apart from this book?</p>

CC BY-SA 4.0
2019-06-18T21:12:16.810Z
Start reading the paper and look up the things that you don't understand or ask a question about it.
Brale
CC BY-SA 4.0
2019-06-20T01:24:36.223Z
After completing the relevant chapter of the book, I would suggest you take a look at the cited references in the paper (especially those parts that you feel you need more materials).
Borhan Kazimipour
CC BY-SA 4.0
2021-10-12T08:50:28.683Z
@Brale yes, this worked for me with GPT-2... however, the tree of unknowns is quite deep!
Criticize SE actions means ban
<reinforcement-learning><papers><alphazero><sutton-barto>
What knowledge is required for understanding the AlphaZero paper?
Laschet Jain
# What knowledge is required for understanding the AlphaZero paper?

(Asked by: username_0 on 2019-06-18)

My goal is to understand AlphaZero paper published by deepmind. I'm beginning my journey trying to get the basic intuition of reinforcement learning from the [book](https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf) by Barto and Sutton.

As per my background, I'm familiar with MDPs, value iteration and policy iteration.

I wanted to ask until what chapter of Barto and Sutton's book is one required to read in order to fully comprehend AlphaZero's paper. Monte-Carlo Tree Search is discussed in Chapter-8 of the book. Will it be enough till that? Or would I be needing more resources apart from this book?

**Tags:** reinforcement-learning, papers, alphazero, sutton-barto

> **Comments:**
> **username_1**: Start reading the paper and look up the things that you don't understand or ask a question about it.  
> **username_2**: After completing the relevant chapter of the book, I would suggest you take a look at the cited references in the paper (especially those parts that you feel you need more materials).  
> **username_3**: @username_1 yes, this worked for me with GPT-2... however, the tree of unknowns is quite deep!  

---

## 1 answers

### Answer by username_4 on 2019-06-19. Score: 4 | Accepted answer

The more you read, the more deeply you can understand any paper, but given your stated background, reading the Monte-Carlo Tree Search chapter of Barto & Sutton, plus Gerald Tesauro's [TD-Gammon paper](https://cling.csd.uwo.ca/cs346a/extra/tdgammon.pdf) (which is pretty accessible, and which is the basis for the *other* technique used in AlphaZero) should be enough to get a pretty good idea of what they did.


---

