Answers.Body: <p>The answer is surprisingly hidden in the original AlphaGo <a href="https://www.nature.com/articles/nature16961" rel="nofollow noreferrer">paper</a>:</p>
  
  <blockquote>
    <p>At the end of search AlphaGo selects the action with maximum visit count; this
    is less sensitive to outliers than maximizing action value.</p>
  </blockquote>
  
  <p>Unfortunately, there did not appear to be further details in the paper or in the related <a href="https://ieeexplore.ieee.org/document/5599855" rel="nofollow noreferrer">reference</a>. The root child node (corresponding to an action) with maximum visit count is fittingly known as the <em>robust child</em>, as described <a href="https://project.dke.maastrichtuniversity.nl/games/files/msc/Fschadd_thesis.pdf" rel="nofollow noreferrer">here</a> and referenced in a MCTS survey <a href="https://ieeexplore.ieee.org/abstract/document/6145622" rel="nofollow noreferrer">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-25T20:31:17.787Z
Answers.Text: One obvious scenario where mean actual value falls short is when the count is small (say, 1).
Answers.UserDisplayName: dshin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-07T04:31:53.620Z
Answers.LastActivityDate: 2020-06-07T04:31:53.620Z
Answers.UserDisplayName: DeepQZero
Body: <p>From <a href="https://deepmind.com/research/publications/mastering-game-go-without-human-knowledge" rel="nofollow noreferrer">the AlphaGo Zero paper</a>, AlphaGo Zero uses an exponentiated visit count from the tree search.</p>
  
  <p>Why use visit count instead of the mean action value <span class="math-container">$Q(s, a)$</span>?</p>
  
Tags: <reinforcement-learning><alphago-zero>
Title: Why does AlphaGo Zero select move based on exponentiated visit count?
UserDisplayName: Cash Lo
ThreadText: # Why does AlphaGo Zero select move based on exponentiated visit count?
  
  (Asked by: username_0 on 2020-06-05)
  
  From [the AlphaGo Zero paper](https://deepmind.com/research/publications/mastering-game-go-without-human-knowledge), AlphaGo Zero uses an exponentiated visit count from the tree search.
  
  Why use visit count instead of the mean action value $Q(s, a)$?
  
  **Tags:** reinforcement-learning, alphago-zero
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-07. Score: 3 | Accepted answer
  
  The answer is surprisingly hidden in the original AlphaGo [paper](https://www.nature.com/articles/nature16961):
  
  > At the end of search AlphaGo selects the action with maximum visit count; this
  > is less sensitive to outliers than maximizing action value.
  
  Unfortunately, there did not appear to be further details in the paper or in the related [reference](https://ieeexplore.ieee.org/document/5599855). The root child node (corresponding to an action) with maximum visit count is fittingly known as the *robust child*, as described [here](https://project.dke.maastrichtuniversity.nl/games/files/msc/Fschadd_thesis.pdf) and referenced in a MCTS survey [here](https://ieeexplore.ieee.org/abstract/document/6145622).
  
  > **Comments:**
  > **username_2**: One obvious scenario where mean actual value falls short is when the count is small (say, 1).  
  
  ---
  
  