Answers.Body: <p>In UCT, the value of Q(vi) / N(vi) is bounded between 0 and 1. Normally when applying MCTS to 2-player games, what happens is the following:<br>
  N(vi) corresponds to the total number of games simulated in node vi.<br>
  Q(vi) corresponds to the total number of games simulated and won in node vi.<br>
  So in each simulation Q(vi) will add +1 to the winning player and +0 to the losing player.  </p>
  
  <p>In a tree representation of a 2-player game, each level will represent player A and player B alternately. <strong>So you will add 1 to every other node</strong>. I don't think it makes sense to add negative values to Qi.</p>
  
  <p>So to answer your question:
  <strong>You only need to store N and Q for each node</strong>, where N is the total number of times that node was in the path of a simulated node and Q is the number of times it won, out of those simulations.</p>
  
  <p>I can give you the example of the way I implemented that part, in a board game I saved in the state class a variable which said who the next player was ( 1 or 2). So I would know that in that state (each node is a state) the player who made the last move was the other one (by the formula: player = 3 - next_player), and I would add 1 to the Qi of those nodes. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-04T13:32:59.637Z
Answers.LastActivityDate: 2019-05-04T13:38:26.523Z
Answers.LastEditDate: 2019-05-04T13:38:26.523Z
Answers.UserDisplayName: Miguel Saraiva
Body: <p>In <a href="https://int8.io/monte-carlo-tree-search-beginners-guide/" rel="nofollow noreferrer">this blog</a> toward the end, the author writes the following:
  <a href="https://i.sstatic.net/j2Dsm.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/j2Dsm.jpg" alt="pic_0"></a>
  <a href="https://i.sstatic.net/fPf77.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/fPf77.jpg" alt="pic"></a></p>
  
  <p>For the sake of my question, let’s assume that a terminal state gives a reward of +1 for a win and -1 for a loss. </p>
  
  <p>When the author says “for any two consecutive nodes this perspective is opposite,” does that mean that if <span class="math-container">$Q_i$</span> is positive (for example, 4) for player A at a given node, the same node will have the negative of that value for player B (-4 in my hypothetical)?</p>
  
  <p>Do I need to compute two statistics to store the node value (one for each player) or can I simply store statistics for one player and flip the sign at every consecutive node?</p>
  
Tags: <monte-carlo-tree-search><implementation>
Title: How does Monte Carlo Tree Search UCT exploitation value change based on perspective?
UserDisplayName: Hanzy
ThreadText: # How does Monte Carlo Tree Search UCT exploitation value change based on perspective?
  
  (Asked by: username_0 on 2019-05-02)
  
  In [this blog](https://int8.io/monte-carlo-tree-search-beginners-guide/) toward the end, the author writes the following:
  [![pic_0](https://i.sstatic.net/j2Dsm.jpg)](https://i.sstatic.net/j2Dsm.jpg)
  [![pic](https://i.sstatic.net/fPf77.jpg)](https://i.sstatic.net/fPf77.jpg)
  
  For the sake of my question, let’s assume that a terminal state gives a reward of +1 for a win and -1 for a loss.
  
  When the author says “for any two consecutive nodes this perspective is opposite,” does that mean that if $Q\_i$ is positive (for example, 4) for player A at a given node, the same node will have the negative of that value for player B (-4 in my hypothetical)?
  
  Do I need to compute two statistics to store the node value (one for each player) or can I simply store statistics for one player and flip the sign at every consecutive node?
  
  **Tags:** monte-carlo-tree-search, implementation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-05-04. Score: 3 | Accepted answer
  
  In UCT, the value of Q(vi) / N(vi) is bounded between 0 and 1. Normally when applying MCTS to 2-player games, what happens is the following:  
  N(vi) corresponds to the total number of games simulated in node vi.  
  Q(vi) corresponds to the total number of games simulated and won in node vi.  
  So in each simulation Q(vi) will add +1 to the winning player and +0 to the losing player.
  
  In a tree representation of a 2-player game, each level will represent player A and player B alternately. **So you will add 1 to every other node**. I don't think it makes sense to add negative values to Qi.
  
  So to answer your question:
  **You only need to store N and Q for each node**, where N is the total number of times that node was in the path of a simulated node and Q is the number of times it won, out of those simulations.
  
  I can give you the example of the way I implemented that part, in a board game I saved in the state class a variable which said who the next player was ( 1 or 2). So I would know that in that state (each node is a state) the player who made the last move was the other one (by the formula: player = 3 - next\_player), and I would add 1 to the Qi of those nodes.
  
  
  ---
  
  