Answers.Body: <p>Tabular Q-Learning does not explicitly create a model of the transition function. It does not generate any output that you can afterwards use as a function to predict what the next state <em>s'</em> will be given a current state <em>s</em> and an action <em>a</em> (that's what a transition function would allow you to do). So no, Q-learning is still model-free.</p>
  
  <p>By the way, model-based RL does not necessarily have to involve creating a model of the transition function. Model-based RL can also mean that you assume that such a function is already given. It just means that you use such a function in some way.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-12-19T11:18:38.760Z
Answers.LastActivityDate: 2019-02-16T19:14:31.143Z
Answers.LastEditDate: 2019-02-16T19:14:31.143Z
Answers.UserDisplayName: Dennis Soemers
Answers.Body: <p>In model-based learning, the learning agent utilizes a model that was previously learned to accomplish a task, whereas model-free RL doesn't use the environment to learn but simply relies on trial and error experience for action selection. The similarity is that in both methods the learning agent is trying to maximize reward from its actions.</p>
  <p>Q-Learning is a model-free RL method. It can be used to identify an optimal action-selection policy for any given finite Markov Decision Process. How it works is that it learns an action value function, which essentially gives the expected utility of an action in a given state, then follows an optimal policy afterwards.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-12-24T21:52:55.987Z
Answers.LastActivityDate: 2022-01-02T09:27:35.223Z
Answers.LastEditDate: 2022-01-02T09:27:35.223Z
Answers.UserDisplayName: Seth Simba
Body: <p>Model-based RL creates a model of the transition function.</p>
  <p>Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a type of model-based RL?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-28T02:27:37.557Z
Comments.Text: I guess, you are talking about the Q table? That is not the same as the transition model.
Comments.UserDisplayName: okkhoy
Tags: <reinforcement-learning><q-learning><definitions><model-based-methods>
Title: Is Q-learning a type of model-based RL?
UserDisplayName: echo
ThreadText: # Is Q-learning a type of model-based RL?
  
  (Asked by: username_0 on 2017-12-19)
  
  Model-based RL creates a model of the transition function.
  
  Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a type of model-based RL?
  
  **Tags:** reinforcement-learning, q-learning, definitions, model-based-methods
  
  > **Comments:**
  > **username_1**: I guess, you are talking about the Q table? That is not the same as the transition model.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2017-12-19. Score: 9
  
  Tabular Q-Learning does not explicitly create a model of the transition function. It does not generate any output that you can afterwards use as a function to predict what the next state *s'* will be given a current state *s* and an action *a* (that's what a transition function would allow you to do). So no, Q-learning is still model-free.
  
  By the way, model-based RL does not necessarily have to involve creating a model of the transition function. Model-based RL can also mean that you assume that such a function is already given. It just means that you use such a function in some way.
  
  
  ---
  
  ### Answer by username_3 on 2017-12-24. Score: 2
  
  In model-based learning, the learning agent utilizes a model that was previously learned to accomplish a task, whereas model-free RL doesn't use the environment to learn but simply relies on trial and error experience for action selection. The similarity is that in both methods the learning agent is trying to maximize reward from its actions.
  
  Q-Learning is a model-free RL method. It can be used to identify an optimal action-selection policy for any given finite Markov Decision Process. How it works is that it learns an action value function, which essentially gives the expected utility of an action in a given state, then follows an optimal policy afterwards.
  
  
  ---
  
  