Answers.Body: <p>It might be the case that if you perform a large number of random rollouts, the "best action" as chosen by the agent without the domain knowledge, is same as the agent with the domain knowledge. I guess what you can do is try to reduce the number of rollouts and see if the performance changes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-17T21:15:29.010Z
Answers.LastActivityDate: 2020-05-17T21:15:29.010Z
Answers.UserDisplayName: pecey
Body: <p>I’ve created an agent using MCTS to play Connect Four. It wins against humans pretty well, but I’d like to improve upon it. I decided to add domain knowledge to the MCTS rollout stage. My evaluation function checks how “good” an action is and returns the best/highest value action to the rollout policy as the action to use. 
  I created a “gym” application for one agent, who’s not using the evaluation function, to play against an agent who is using the evaluation function. 
  I would have expected the agent using the heuristics to perform better than the agent who isn’t, but the inclusion of the heuristics doesn’t seem to make any difference! Any ideas why this might be the case? </p>
  
Tags: <reinforcement-learning><monte-carlo-tree-search><heuristics>
Title: Why aren’t heuristics for Connect Four Monte Carlo tree search improving the agent?
UserDisplayName: mason7663
ThreadText: # Why aren’t heuristics for Connect Four Monte Carlo tree search improving the agent?
  
  (Asked by: username_0 on 2020-05-16)
  
  I’ve created an agent using MCTS to play Connect Four. It wins against humans pretty well, but I’d like to improve upon it. I decided to add domain knowledge to the MCTS rollout stage. My evaluation function checks how “good” an action is and returns the best/highest value action to the rollout policy as the action to use.
  I created a “gym” application for one agent, who’s not using the evaluation function, to play against an agent who is using the evaluation function.
  I would have expected the agent using the heuristics to perform better than the agent who isn’t, but the inclusion of the heuristics doesn’t seem to make any difference! Any ideas why this might be the case?
  
  **Tags:** reinforcement-learning, monte-carlo-tree-search, heuristics
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-17. Score: 1
  
  It might be the case that if you perform a large number of random rollouts, the "best action" as chosen by the agent without the domain knowledge, is same as the agent with the domain knowledge. I guess what you can do is try to reduce the number of rollouts and see if the performance changes.
  
  
  ---
  
  