Answers.Body: <p>I also had the same question, but after looking at this two links:
  <a href="https://smartlabai.medium.com/a-brief-overview-of-imitation-learning-8a8a75c44a9c" rel="nofollow noreferrer">this article</a> and <a href="https://web.stanford.edu/class/cs237b/pdfs/lecture/lecture_10111213.pdf" rel="nofollow noreferrer">this lecture</a> I think we can say that behavioral cloning (which is the simplest way for doing imitation learning) is just normal supervised learning. But imitation learning could be associated with RL because it has other types (improvements) that involve learning while interacting with the environment like Dataset Aggregation, where you have an interactive expert that you can query for optimal actions to label observations obtained by rolling out the learned policy in the environment, and you repeat collecting the data and training the policy. This is done to mitigate the known problem of distributional drift in BC. More about it can be found <a href="https://web.stanford.edu/class/cs237b/pdfs/lecture/lecture_10111213.pdf" rel="nofollow noreferrer">here</a>.</p>
  <p>Even with this interactive expert, I still think it is supervised learning, but I can understand why some people might associate it with RL.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-25T18:01:40.093Z
Answers.Text: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-25T17:07:43.253Z
Answers.LastActivityDate: 2022-11-25T19:41:44.277Z
Answers.LastEditDate: 2022-11-25T19:41:44.277Z
Answers.UserDisplayName: Abdalwhab Bakheet
Answers.Body: <p>Imitation learning <em>is</em> supervised learning applied to the RL setting.</p>
  
  <p>In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a scenario where you have available the optimal policy in the form of a table, mapping each state to each action. In this scenario you will not care about the rewards received - rather, you'd simply do a table lookup to decide the optimal action. </p>
  
  <p>This scenario is impractical in most settings because the table for the optimal policy will be too big. However, if you have enough entries from the table, you can use a general function approximator such as a neural network to find the optimal action. Again, you do not need to look at the rewards, but only at the state <span class="math-container">$\rightarrow$</span> action mappings. I do not know imitation learning in detail beyond this, but I suspect in the case of discrete actions (such as in Chess, Go), it would be trained with a cross-entropy objective as is typical of classification tasks.</p>
  
  <p>I suspect that the reason this has a different name in the RL setting is because this is different than how the conventional RL algorithms work. Also, much of RL thinking is inspired from everyday interaction / biology. Think of how we learn how to drive, or play sports such as soccer. Typically there is a coach who tells you what actions to take under different conditions, and you imitate those actions. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-19T07:20:51.500Z
Answers.LastActivityDate: 2018-12-19T07:20:51.500Z
Answers.UserDisplayName: Sabyasachi Ghosh
Body: <p>In short, imitation learning means learning from the experts. Suppose I have a dataset with labels based on the actions of experts. I use a simple binary classifier algorithm to assess whether it is good expert action or bad expert action.</p>
  <p>How is this binary classification different from imitation learning?</p>
  <p>Imitation learning is associated with reinforcement learning, but, in this case, it looks more like a basic classification problem to me.</p>
  <p>What is the difference between imitation learning and classification done by experts?</p>
  <p>I am getting confused because imitation learning relates to reinforcement learning while classification relates to supervised learning.</p>
  
Tags: <reinforcement-learning><classification><comparison><supervised-learning><imitation-learning>
Title: What is the difference between imitation learning and classification done by experts?
UserDisplayName: user781486
ThreadText: # What is the difference between imitation learning and classification done by experts?
  
  (Asked by: username_0 on 2018-12-19)
  
  In short, imitation learning means learning from the experts. Suppose I have a dataset with labels based on the actions of experts. I use a simple binary classifier algorithm to assess whether it is good expert action or bad expert action.
  
  How is this binary classification different from imitation learning?
  
  Imitation learning is associated with reinforcement learning, but, in this case, it looks more like a basic classification problem to me.
  
  What is the difference between imitation learning and classification done by experts?
  
  I am getting confused because imitation learning relates to reinforcement learning while classification relates to supervised learning.
  
  **Tags:** reinforcement-learning, classification, comparison, supervised-learning, imitation-learning
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2018-12-19. Score: 5 | Accepted answer
  
  Imitation learning *is* supervised learning applied to the RL setting.
  
  In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a scenario where you have available the optimal policy in the form of a table, mapping each state to each action. In this scenario you will not care about the rewards received - rather, you'd simply do a table lookup to decide the optimal action.
  
  This scenario is impractical in most settings because the table for the optimal policy will be too big. However, if you have enough entries from the table, you can use a general function approximator such as a neural network to find the optimal action. Again, you do not need to look at the rewards, but only at the state $\rightarrow$ action mappings. I do not know imitation learning in detail beyond this, but I suspect in the case of discrete actions (such as in Chess, Go), it would be trained with a cross-entropy objective as is typical of classification tasks.
  
  I suspect that the reason this has a different name in the RL setting is because this is different than how the conventional RL algorithms work. Also, much of RL thinking is inspired from everyday interaction / biology. Think of how we learn how to drive, or play sports such as soccer. Typically there is a coach who tells you what actions to take under different conditions, and you imitate those actions.
  
  
  ---
  
  ### Answer by username_1 on 2022-11-25. Score: 2
  
  I also had the same question, but after looking at this two links:
  [this article](https://smartlabai.medium.com/a-brief-overview-of-imitation-learning-8a8a75c44a9c) and [this lecture](https://web.stanford.edu/class/cs237b/pdfs/lecture/lecture_10111213.pdf) I think we can say that behavioral cloning (which is the simplest way for doing imitation learning) is just normal supervised learning. But imitation learning could be associated with RL because it has other types (improvements) that involve learning while interacting with the environment like Dataset Aggregation, where you have an interactive expert that you can query for optimal actions to label observations obtained by rolling out the learned policy in the environment, and you repeat collecting the data and training the policy. This is done to mitigate the known problem of distributional drift in BC. More about it can be found [here](https://web.stanford.edu/class/cs237b/pdfs/lecture/lecture_10111213.pdf).
  
  Even with this interactive expert, I still think it is supervised learning, but I can understand why some people might associate it with RL.
  
  > **Comments:**
  > **username_2**: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  
  ---
  
  