Answers.Body: <p>Disclaimer, not an expert.</p>
  
  <p>As I understand it, the process runs something like this:</p>
  
  <ul>
  <li>Given current position, ask the neural network for a prediction on 1) the move and 2) the win probability arising from that move.</li>
  <li>Make that move. This gets to a leaf node. Query neural network for new prediction on 1) the move and 2) the win probability arising from that move.</li>
  <li>And so on ...</li>
  <li>Because it's Monte Carlo Tree Search, at some points the program will back up and search the move with the next-highest win probability.</li>
  <li>When time is up, make the move that leads to the highest win probability.</li>
  </ul>
  
  <p>See <a href="https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png" rel="nofollow noreferrer">this link</a> for an infographic.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-08T08:55:32.470Z
Answers.Text: As far as I understand, it uses the move it explored the most, since moves that have a high win probability may not be explored enough to see if they have refutations
Answers.UserDisplayName: iopq
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-22T13:17:15.783Z
Answers.LastActivityDate: 2019-02-22T13:17:15.783Z
Answers.UserDisplayName: Allure
Answers.Body: <p>AlphaZero and Lc0 use a modified version of MCTS.</p>
  <p>The neural networks in A0 and Lc0 are actually &quot;two-headed&quot; networks - one is a &quot;value network,&quot; and the other is a &quot;policy network.&quot; The value network is very simple: Given a position, it provides a probability that either side will win. Note that it is not based on color - in Lc0, 1 means that the side to move is 100% winning, and -1 means that the side that is not moving is 100% winning. The policy network, given a position, produces probabilities for each move being played.</p>
  <p>Interestingly enough, the network gives probabilities for all theoretically legal moves, treating each piece as a &quot;superpiece&quot; that can move as both a queen and a knight. Illegal moves are filtered out, their probabilities set to 0 and the probabilities for the othre moves recalculated. In this way, the outputs can of fixed size.</p>
  <p>At the beginning of MCTS, you select nodes based on the UCT value, which is a formula that balances &quot;Explotation&quot; (picking nodes you know are pretty good) versus &quot;Exploration&quot; (picking nodes that you haven't seen before.) These engines use a variant called &quot;PUCT,&quot; or predictor UCT, which takes into account the policy network's suggestions. Then, rather than using rollouts, the network just uses the evaluation from the value network.</p>
  <p>Dominik Klein wrote a really great book about neural networks in chess - I highly recommend you give it a read! The link is here:
  <a href="https://github.com/asdfjkl/neural_network_chess" rel="nofollow noreferrer">https://github.com/asdfjkl/neural_network_chess</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-09T23:21:40.927Z
Answers.LastActivityDate: 2022-07-09T23:21:40.927Z
Answers.UserDisplayName: AAce3
Answers.Body: <p>Neither AlphaZero, nor Lc0 (an open source project based on the AlphaZero paper), use MCTS in its pure form, because they do not perform game rollouts - the search is simply called MCTS because the way that it expands its tree is similar to MCTS.</p>
  
  <p>Here are the main differences:</p>
  
  <ul>
  <li>MCTS performs random rollouts to a termination condition (e.g. win, draw, loss) at leaf nodes to estimate to value of that node, whereas AlphaZero and Lc0 simply use the neural network to estimate the win value. They do not perform rollouts.</li>
  <li>MCTS uses UCT to decide which node should be expanded. AlphaZero and Lc0 use the output of the neural network as an additional parameter in their versions of UCT: the neural network generates a probability associated with every possible move in the position, which allows the neural network to control the expansion of the search tree to some degree.</li>
  </ul>
  
  <p><a href="https://github.com/LeelaChessZero/lc0" rel="noreferrer">Lc0</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-21T20:35:45.780Z
Answers.LastActivityDate: 2019-02-21T20:35:45.780Z
Answers.UserDisplayName: konsolas
Body: <p>Where does an engine (such as AlphaZero) use its neural network in MCTS? It shouldn't be needed to evaluate positions reached via MCTS, since I'm under the impression that only terminal game states are evaluated (checkmate or draws). Is the neural network used in the expansion portion of the MCTS algorithm? I.e., choosing which new move to expand on a leaf node, rather than randomly choosing a new move? If that's the case then it doesn't seem like the neural network plays such an important role.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-21T18:44:29.990Z
Comments.Text: The neural network plays the evaluation part of a traditional engine.
Comments.UserDisplayName: Wais Kamal
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-21T20:55:56.583Z
Comments.Text: For AlphaZero, have a look at their officially published paper ([open access version here](http://science.sciencemag.org/content/362/6419/1140.full?ijkey=XGd77kI6W4rSc&keytype=ref&siteid=sci)), it's clear on these details. See the MCTS discussion at the beginning, then Fig. 4 in particular and the discussions thereof.
Comments.UserDisplayName: Ellie
Tags: <engines><programming><alphazero>
Title: Where is the neural network used in MCTS?
ThreadText: # Where is the neural network used in MCTS?
  
  (Asked by: username_0 on 2019-02-21)
  
  Where does an engine (such as AlphaZero) use its neural network in MCTS? It shouldn't be needed to evaluate positions reached via MCTS, since I'm under the impression that only terminal game states are evaluated (checkmate or draws). Is the neural network used in the expansion portion of the MCTS algorithm? I.e., choosing which new move to expand on a leaf node, rather than randomly choosing a new move? If that's the case then it doesn't seem like the neural network plays such an important role.
  
  **Tags:** engines, programming, alphazero
  
  > **Comments:**
  > **username_1**: The neural network plays the evaluation part of a traditional engine.  
  > **username_2**: For AlphaZero, have a look at their officially published paper ([open access version here](http://science.sciencemag.org/content/362/6419/1140.full?ijkey=XGd77kI6W4rSc&keytype=ref&siteid=sci)), it's clear on these details. See the MCTS discussion at the beginning, then Fig. 4 in particular and the discussions thereof.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_6 on 2019-02-21. Score: 6 | Accepted answer
  
  Neither AlphaZero, nor Lc0 (an open source project based on the AlphaZero paper), use MCTS in its pure form, because they do not perform game rollouts - the search is simply called MCTS because the way that it expands its tree is similar to MCTS.
  
  Here are the main differences:
  
  * MCTS performs random rollouts to a termination condition (e.g. win, draw, loss) at leaf nodes to estimate to value of that node, whereas AlphaZero and Lc0 simply use the neural network to estimate the win value. They do not perform rollouts.
  * MCTS uses UCT to decide which node should be expanded. AlphaZero and Lc0 use the output of the neural network as an additional parameter in their versions of UCT: the neural network generates a probability associated with every possible move in the position, which allows the neural network to control the expansion of the search tree to some degree.
  
  [Lc0](https://github.com/LeelaChessZero/lc0)
  
  
  ---
  
  ### Answer by username_3 on 2019-02-22. Score: 3
  
  Disclaimer, not an expert.
  
  As I understand it, the process runs something like this:
  
  * Given current position, ask the neural network for a prediction on 1) the move and 2) the win probability arising from that move.
  * Make that move. This gets to a leaf node. Query neural network for new prediction on 1) the move and 2) the win probability arising from that move.
  * And so on ...
  * Because it's Monte Carlo Tree Search, at some points the program will back up and search the move with the next-highest win probability.
  * When time is up, make the move that leads to the highest win probability.
  
  See [this link](https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png) for an infographic.
  
  > **Comments:**
  > **username_4**: As far as I understand, it uses the move it explored the most, since moves that have a high win probability may not be explored enough to see if they have refutations  
  
  ---
  
  ### Answer by username_5 on 2022-07-09. Score: 1
  
  AlphaZero and Lc0 use a modified version of MCTS.
  
  The neural networks in A0 and Lc0 are actually "two-headed" networks - one is a "value network," and the other is a "policy network." The value network is very simple: Given a position, it provides a probability that either side will win. Note that it is not based on color - in Lc0, 1 means that the side to move is 100% winning, and -1 means that the side that is not moving is 100% winning. The policy network, given a position, produces probabilities for each move being played.
  
  Interestingly enough, the network gives probabilities for all theoretically legal moves, treating each piece as a "superpiece" that can move as both a queen and a knight. Illegal moves are filtered out, their probabilities set to 0 and the probabilities for the othre moves recalculated. In this way, the outputs can of fixed size.
  
  At the beginning of MCTS, you select nodes based on the UCT value, which is a formula that balances "Explotation" (picking nodes you know are pretty good) versus "Exploration" (picking nodes that you haven't seen before.) These engines use a variant called "PUCT," or predictor UCT, which takes into account the policy network's suggestions. Then, rather than using rollouts, the network just uses the evaluation from the value network.
  
  Dominik Klein wrote a really great book about neural networks in chess - I highly recommend you give it a read! The link is here:
  <https://github.com/asdfjkl/neural_network_chess>
  
  
  ---
  
  