Answers.Body: <blockquote>
    <p>If we choose to only expand a single node, the nodes that were not expanded will never be considered in future selections as we only select child nodes until a leaf is reached during the Selection Phase. Is this correct? </p>
  </blockquote>
  
  <p>No this is not correct.</p>
  
  <blockquote>
    <p>If not, what am I misunderstanding about the Selection Phase?</p>
  </blockquote>
  
  <p>The selection phase does not end only when you reach a node that has no expanded nodes. It ends when you reach a node that has <em>any</em> unexpanded nodes. At which point you typically pick one or more nodes you have not yet expanded at that point in the tree, expand them and collect one or more rollout results for them. Variations are possible such as choosing to whether to expand or continue selecting stochastically, or expanding all child nodes at the same time using value estimates to initialise them - the latter is what AlphaZero does.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-13T10:20:39.640Z
Answers.Text: "pick one or more nodes" ... From my reading on MCTS the "picking" is conducted by the Tree Policy which is the argmax of action-values (in some form). The problem when one only expands one node at a time is: how is the argmax supposed to be computed? You at the very least need a placeholder for ALL the action-values of child nodes for the Tree Policy to choose from. Hence it makes sense to always expand all nodes from a selected node in theory (practically there are reasons not to do this e.g. memory usage)
Answers.UserDisplayName: Johan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-18T06:39:47.347Z
Answers.LastActivityDate: 2019-06-18T08:32:42.257Z
Answers.LastEditDate: 2019-06-18T08:32:42.257Z
Answers.UserDisplayName: Neil Slater
Body: <p>All sources I can find provide a similar explanation to each phase. </p>
  
  <p>In the Selection Phase, we start at the root and choose child nodes until reaching a leaf.  Once the leaf is reached (assuming the game is not terminated), we enter the Expansion Phase. </p>
  
  <p>In the Expansion Phase, we expand any number of child nodes and select one of the expanded nodes. Then, we enter the Play-Out Phase.</p>
  
  <p>Here is my confusion. If we choose to only expand a single node, the nodes that were not expanded will never be considered in future selections as we only select child nodes until a leaf is reached during the Selection Phase. Is this correct? If not, what am I misunderstanding about the Selection Phase?</p>
  
Tags: <game-ai><monte-carlo-tree-search>
Title: When does the selection phase exactly end in MCTS?
UserDisplayName: Ralff
ThreadText: # When does the selection phase exactly end in MCTS?
  
  (Asked by: username_0 on 2019-06-18)
  
  All sources I can find provide a similar explanation to each phase.
  
  In the Selection Phase, we start at the root and choose child nodes until reaching a leaf. Once the leaf is reached (assuming the game is not terminated), we enter the Expansion Phase.
  
  In the Expansion Phase, we expand any number of child nodes and select one of the expanded nodes. Then, we enter the Play-Out Phase.
  
  Here is my confusion. If we choose to only expand a single node, the nodes that were not expanded will never be considered in future selections as we only select child nodes until a leaf is reached during the Selection Phase. Is this correct? If not, what am I misunderstanding about the Selection Phase?
  
  **Tags:** game-ai, monte-carlo-tree-search
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-18. Score: 2 | Accepted answer
  
  > If we choose to only expand a single node, the nodes that were not expanded will never be considered in future selections as we only select child nodes until a leaf is reached during the Selection Phase. Is this correct?
  
  No this is not correct.
  
  > If not, what am I misunderstanding about the Selection Phase?
  
  The selection phase does not end only when you reach a node that has no expanded nodes. It ends when you reach a node that has *any* unexpanded nodes. At which point you typically pick one or more nodes you have not yet expanded at that point in the tree, expand them and collect one or more rollout results for them. Variations are possible such as choosing to whether to expand or continue selecting stochastically, or expanding all child nodes at the same time using value estimates to initialise them - the latter is what AlphaZero does.
  
  > **Comments:**
  > **username_2**: "pick one or more nodes" ... From my reading on MCTS the "picking" is conducted by the Tree Policy which is the argmax of action-values (in some form). The problem when one only expands one node at a time is: how is the argmax supposed to be computed? You at the very least need a placeholder for ALL the action-values of child nodes for the Tree Policy to choose from. Hence it makes sense to always expand all nodes from a selected node in theory (practically there are reasons not to do this e.g. memory usage)  
  
  ---
  
  