<p>In the vanilla <a href="https://en.wikipedia.org/wiki/Monte_Carlo_tree_search" rel="nofollow noreferrer">Monte Carlo tree search (MCTS)</a> implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only then the information gathered is backed up.</p>
<p>I have read the AlphaZero paper (and the AlphaGo Zero too) and I didn't find any information on how the rollout is implemented (maybe I missed it).</p>
<p>How is the rollout from the MCTS implemented in both the AlphaGo Zero and the AlphaZero algorithms?</p>

CC BY-SA 4.0
2019-11-03T07:40:06.163Z
AlphaZero/Go don't have rollout. Rollout replaced with neural network value estimation
mirror2image
CC BY-SA 4.0
2019-11-03T09:03:45.110Z
@mi: I thought that original Alpha Go *did* have a rollout policy, but that one of the simplifying changes in AlphaZero was to completely remove it?
Neil Slater
CC BY-SA 4.0
2019-11-03T13:53:07.980Z
@NeilSlater That's correct. The question seems to be about AlphaGo Zero and AlphaZero though, not about Alpha Go :) **AlphaGo**: uses a fast rollout policy (trained like the policy network, but is not a large DNN, just a single layer + softmax I believe). **AlphaGo Zero**: no rollouts. **AlphaZero**: no rollouts.
Dennis Soemers
CC BY-SA 4.0
2019-11-03T17:57:52.330Z
@mirror2image even though it was only a line, you answered the question -- why not make it slightly more verbose and make it an answer
mshlis
CC BY-SA 4.0
2019-11-03T23:18:40.597Z
Thanks for the answers. However, I'm still confused. I understand that they use the V value returned from the network to simulate who won, but how do they know which value of V a player won or lost? Because as far as I understand, in the weight update (in the loss function), they will compare the probability distribution gathered from the self-plays and the difference between the real winner (from MCTS) and the value V.
ihavenoidea
CC BY-SA 4.0
2019-11-03T23:19:30.217Z
In short, I'm confused about the V value range returned from the network. Will it be in the [-1,1] range?
ihavenoidea
CC BY-SA 4.0
2021-07-29T19:16:42.040Z
@ihavenoidea  I got the same question have you figured it out?
Daniel
<monte-carlo-tree-search><alphazero><implementation><alphago-zero>
How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?
ihavenoidea
# How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?

(Asked by: username_0 on 2019-11-03)

In the vanilla [Monte Carlo tree search (MCTS)](https://en.wikipedia.org/wiki/Monte_Carlo_tree_search) implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only then the information gathered is backed up.

I have read the AlphaZero paper (and the AlphaGo Zero too) and I didn't find any information on how the rollout is implemented (maybe I missed it).

How is the rollout from the MCTS implemented in both the AlphaGo Zero and the AlphaZero algorithms?

**Tags:** monte-carlo-tree-search, alphazero, implementation, alphago-zero

> **Comments:**
> **username_1**: AlphaZero/Go don't have rollout. Rollout replaced with neural network value estimation  
> **username_2**: @mi: I thought that original Alpha Go *did* have a rollout policy, but that one of the simplifying changes in AlphaZero was to completely remove it?  
> **username_3**: @NeilSlater That's correct. The question seems to be about AlphaGo Zero and AlphaZero though, not about Alpha Go :) **AlphaGo**: uses a fast rollout policy (trained like the policy network, but is not a large DNN, just a single layer + softmax I believe). **AlphaGo Zero**: no rollouts. **AlphaZero**: no rollouts.  
> **username_4**: @username_1 even though it was only a line, you answered the question -- why not make it slightly more verbose and make it an answer  
> **username_0**: Thanks for the answers. However, I'm still confused. I understand that they use the V value returned from the network to simulate who won, but how do they know which value of V a player won or lost? Because as far as I understand, in the weight update (in the loss function), they will compare the probability distribution gathered from the self-plays and the difference between the real winner (from MCTS) and the value V.  
> **username_0**: In short, I'm confused about the V value range returned from the network. Will it be in the [-1,1] range?  
> **username_5**: @username_0  I got the same question have you figured it out?  

---

## 0 answers

