Answers.Body: <p>You either need a generative model or an emulator of the environment. In the later case you don't calculate your transitions and rewards using the model but feed your actions and states to the emulator and work with the results.</p>
  
  <p>The emulator can be a black box as long as it returns the next state and the reward when provided with the current state and an action. You also need a way to identify all legal actions in a given state to build the tree.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-30T22:28:14.747Z
Answers.LastActivityDate: 2017-10-30T22:28:14.747Z
Answers.UserDisplayName: Demento
Body: <p>From what I have understood reading the UCT paper <a href="http://ggp.stanford.edu/readings/uct.pdf" rel="nofollow noreferrer">Bandit based monte-carlo planning</a>, by Levente Kocsis and Csaba Szepesvári, MCTS/UCT requires a generative model. </p>
  
  <ol>
  <li><p>Does it mean that, in case there is no generative model of the environment, we cannot use MCTS?</p></li>
  <li><p>If we can still use MCTS, how does the roll-out happen in this case, as there is no simulation? </p></li>
  </ol>
  
Tags: <reinforcement-learning><monte-carlo-tree-search>
Title: Can we use MCTS without a generative model?
UserDisplayName: okkhoy
ThreadText: # Can we use MCTS without a generative model?
  
  (Asked by: username_0 on 2017-05-09)
  
  From what I have understood reading the UCT paper [Bandit based monte-carlo planning](http://ggp.stanford.edu/readings/uct.pdf), by Levente Kocsis and Csaba Szepesvári, MCTS/UCT requires a generative model.
  
  1. Does it mean that, in case there is no generative model of the environment, we cannot use MCTS?
  2. If we can still use MCTS, how does the roll-out happen in this case, as there is no simulation?
  
  **Tags:** reinforcement-learning, monte-carlo-tree-search
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-10-30. Score: 3 | Accepted answer
  
  You either need a generative model or an emulator of the environment. In the later case you don't calculate your transitions and rewards using the model but feed your actions and states to the emulator and work with the results.
  
  The emulator can be a black box as long as it returns the next state and the reward when provided with the current state and an action. You also need a way to identify all legal actions in a given state to build the tree.
  
  
  ---
  
  