Answers.Body: <p>That's up to you and mainly depends on your budget:</p>
  <ul>
  <li>if you want it to be as cheap as possible, you probably want to forward the state on the policy network and pick the move with the highest probability (only need one forward pass)</li>
  <li>if you want to be a bit stronger, you can do a 1 step look ahead by either trying all possible moves and evaluating the state you end up being (chess has a branching factor of around 35), or you can just pick the top-k actions from the policy network (needs in the 10s of forward passes)</li>
  <li>at the end, you have the full MCTS. At this point, it heavily depends on how much computation you want to spend... based on that, you can decide both the &quot;breadth&quot; (top K moves for each state) and the &quot;depth&quot; (how many moves forward to simulate). At the end, you usually just pick the move that leads you to the state with highest value (probability of winning)</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-19T22:05:19.277Z
Answers.Text: To make sure I understand.  Let's say breadth is full and depth is 2 or some other constant. Are you saying that given a board position, you construct the full tree of possibilities of depth 2 (like 2 moves), use the neural network to evaluate each position and thus put weights to the leaves, and then find the best moves from the root based on that?
Answers.UserDisplayName: Manu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-19T22:06:24.367Z
Answers.Text: My question is: Let's say you keep the depth to very small, like 1 or 2.  How strong would such an engine be?  Would it be competitive, say 2000 rating or above?  Or do you need deeper trees for that?
Answers.UserDisplayName: Manu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-19T22:14:56.553Z
Answers.Text: @Manu given the specifications in your question, yes
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-19T22:16:04.520Z
Answers.Text: @Manu about the second one, depends on the value network fitting. If it was perfect, then even just 1 step would be enough, though since it's not, you add some depth in the hope that you get a much more accurate estimate
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-19T22:35:26.073Z
Answers.Text: about the second one.  Thanks. My question is, is there such an engine with small depth that achieves decent performance?  For example, what is the depth in alphazero and other game engines?  (I also have to say that I don't quite understand why doing a tree gives a more accurate estimate: if the evaluation of the leaves is poor, I don't get how a tree is fixing that, though maybe I don't understand how it's used.)
Answers.UserDisplayName: Manu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-19T22:59:30.227Z
Answers.Text: @Manu you get a more accurate result because (theoretically) you have "less possible futures to account for", and thus less variance... about alphago, afaik, they fix as budgets the moves, and then using some heuristics, explore the tree with such budget (something like, I explore this state proportionally of how good it is compared to the other ones that I can explore), but searching on the internet you can find some forums speculating that usually it rarely gets over 20 (but it's mere speculation at this point, as the computational budget is expressed in actions, and not depth)
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-19T23:03:47.683Z
Answers.Text: Thanks. Sorry, do you know of any engine out there with a fixed depth bound?
Answers.UserDisplayName: Manu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-20T10:32:58.037Z
Answers.Text: @Manu no, probably because it’s better to use action evaluations as budget rather than just depth
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-19T19:50:43.000Z
Answers.LastActivityDate: 2024-05-19T19:50:43.000Z
Answers.UserDisplayName: Alberto
Body: <p>After the network is trained, how is a move selected in various neural-network chess engines?</p>
  <p>I have seen several options like, go through every possible move and pick the one scored highest by the network.  On the other hand, the alpha zero <a href="https://arxiv.org/abs/1712.01815" rel="nofollow noreferrer">paper</a> talks about various forms of tree search. It is not clear to me how they are used to play, and from the appendix apparently some other engines use very limited search, like depth 2.</p>
  <p>This is a shortened version of <a href="https://ai.stackexchange.com/questions/45730/structure-and-parameters-of-modern-ai-baed-chess-engines">Structure and parameters of modern AI-baed chess engines</a></p>
  
Tags: <neural-networks><chess>
Title: How do neural-network chess engines select moves?
UserDisplayName: Manu
ThreadText: # How do neural-network chess engines select moves?
  
  (Asked by: username_0 on 2024-05-19)
  
  After the network is trained, how is a move selected in various neural-network chess engines?
  
  I have seen several options like, go through every possible move and pick the one scored highest by the network. On the other hand, the alpha zero [paper](https://arxiv.org/abs/1712.01815) talks about various forms of tree search. It is not clear to me how they are used to play, and from the appendix apparently some other engines use very limited search, like depth 2.
  
  This is a shortened version of [Structure and parameters of modern AI-baed chess engines](https://ai.stackexchange.com/questions/45730/structure-and-parameters-of-modern-ai-baed-chess-engines)
  
  **Tags:** neural-networks, chess
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-05-19. Score: 1
  
  That's up to you and mainly depends on your budget:
  
  * if you want it to be as cheap as possible, you probably want to forward the state on the policy network and pick the move with the highest probability (only need one forward pass)
  * if you want to be a bit stronger, you can do a 1 step look ahead by either trying all possible moves and evaluating the state you end up being (chess has a branching factor of around 35), or you can just pick the top-k actions from the policy network (needs in the 10s of forward passes)
  * at the end, you have the full MCTS. At this point, it heavily depends on how much computation you want to spend... based on that, you can decide both the "breadth" (top K moves for each state) and the "depth" (how many moves forward to simulate). At the end, you usually just pick the move that leads you to the state with highest value (probability of winning)
  
  > **Comments:**
  > **username_0**: To make sure I understand.  Let's say breadth is full and depth is 2 or some other constant. Are you saying that given a board position, you construct the full tree of possibilities of depth 2 (like 2 moves), use the neural network to evaluate each position and thus put weights to the leaves, and then find the best moves from the root based on that?  
  > **username_0**: My question is: Let's say you keep the depth to very small, like 1 or 2.  How strong would such an engine be?  Would it be competitive, say 2000 rating or above?  Or do you need deeper trees for that?  
  > **username_1**: @username_0 given the specifications in your question, yes  
  > **username_1**: @username_0 about the second one, depends on the value network fitting. If it was perfect, then even just 1 step would be enough, though since it's not, you add some depth in the hope that you get a much more accurate estimate  
  > **username_0**: about the second one.  Thanks. My question is, is there such an engine with small depth that achieves decent performance?  For example, what is the depth in alphazero and other game engines?  (I also have to say that I don't quite understand why doing a tree gives a more accurate estimate: if the evaluation of the leaves is poor, I don't get how a tree is fixing that, though maybe I don't understand how it's used.)  
  > **username_1**: @username_0 you get a more accurate result because (theoretically) you have "less possible futures to account for", and thus less variance... about alphago, afaik, they fix as budgets the moves, and then using some heuristics, explore the tree with such budget (something like, I explore this state proportionally of how good it is compared to the other ones that I can explore), but searching on the internet you can find some forums speculating that usually it rarely gets over 20 (but it's mere speculation at this point, as the computational budget is expressed in actions, and not depth)  
  > **username_0**: Thanks. Sorry, do you know of any engine out there with a fixed depth bound?  
  > **username_1**: @username_0 no, probably because it’s better to use action evaluations as budget rather than just depth  
  
  ---
  
  