Answers.Body: <p>Traditionally (when not considering your idea), the evaluation function for terminal game states would be implemented to return <span class="math-container">$1$</span>, <span class="math-container">$0$</span>, or <span class="math-container">$-1$</span> for wins, draws, or losses, respectively.</p>
  
  <p>Changing that in a naive/straightforward way to make short-term wins more rewarding, long-term wins less rewarding, short-term losses more negative, and long-term losses less negative can be dangerous, it may change the objective that your agent is ultimately optimizing for (i.e. may lose the guarantee of converging towards optimal play given an infinite amount of time) if not done very carefully.</p>
  
  <p>There is definitely value in considering the idea though, especially because in the Play-Out phase of MCTS, trajectories of (semi-)random moves introduce uncertainty in the evaluations at the end of those simulations, and this uncertainty increases as the length of the trajectories increases (due to increased number of uninformed decisions being made along the trajectory). Note that it is especially important to take into consideration here the number of moves played in the Play-Out phase, not necessarily including the number of moves made in the Selection phase (which are selected according to a much more informed strategy).</p>
  
  <p>One paper I know of that investigates ideas along these lines is <a href="https://dke.maastrichtuniversity.nl/m.winands/documents/QB_ECAI2014.pdf" rel="nofollow noreferrer">"Quality-based Rewards for Monte-Carlo Tree Search Simulations"</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-07T19:46:39.377Z
Answers.LastActivityDate: 2018-10-07T19:46:39.377Z
Answers.UserDisplayName: Dennis Soemers
Body: <p>I'm programming on Connect6 with MCTS.</p>
  
  <p>Monte Carlo Tree Search is based on random moves. It counts up the number of wins in certain moves. (Whether it wins in 3 turns or 30 turns)</p>
  
  <p>Is the move with less turns more powerful than the move with more turns?(as mcts just sees if it's win or not -- not considering the number of turns it took to win) And if so, is it meaningful to give bigger weight to the one with less turn win?</p>
  
Tags: <game-ai><monte-carlo-tree-search>
Title: Is it meaningful to give more weight to the result of monte carlo search with less turn win?
UserDisplayName: kim
ThreadText: # Is it meaningful to give more weight to the result of monte carlo search with less turn win?
  
  (Asked by: username_0 on 2018-10-07)
  
  I'm programming on Connect6 with MCTS.
  
  Monte Carlo Tree Search is based on random moves. It counts up the number of wins in certain moves. (Whether it wins in 3 turns or 30 turns)
  
  Is the move with less turns more powerful than the move with more turns?(as mcts just sees if it's win or not -- not considering the number of turns it took to win) And if so, is it meaningful to give bigger weight to the one with less turn win?
  
  **Tags:** game-ai, monte-carlo-tree-search
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-10-07. Score: 2
  
  Traditionally (when not considering your idea), the evaluation function for terminal game states would be implemented to return $1$, $0$, or $-1$ for wins, draws, or losses, respectively.
  
  Changing that in a naive/straightforward way to make short-term wins more rewarding, long-term wins less rewarding, short-term losses more negative, and long-term losses less negative can be dangerous, it may change the objective that your agent is ultimately optimizing for (i.e. may lose the guarantee of converging towards optimal play given an infinite amount of time) if not done very carefully.
  
  There is definitely value in considering the idea though, especially because in the Play-Out phase of MCTS, trajectories of (semi-)random moves introduce uncertainty in the evaluations at the end of those simulations, and this uncertainty increases as the length of the trajectories increases (due to increased number of uninformed decisions being made along the trajectory). Note that it is especially important to take into consideration here the number of moves played in the Play-Out phase, not necessarily including the number of moves made in the Selection phase (which are selected according to a much more informed strategy).
  
  One paper I know of that investigates ideas along these lines is ["Quality-based Rewards for Monte-Carlo Tree Search Simulations"](https://dke.maastrichtuniversity.nl/m.winands/documents/QB_ECAI2014.pdf).
  
  
  ---
  
  