Answers.Body: <p>Imitation learning data usually means data gathered from an expert, that is data from an agent proficient in the task.</p>
  <p>The agent may be:</p>
  <ul>
  <li>A human operator: have the operator complete the task and record inputs and action taken.</li>
  <li>A pre-trained reinforcement learning agent. Same as above.</li>
  </ul>
  <p>The collected data is exactly the same for both cases.</p>
  <p>The hard part is usually building an interface to collect this data, which is very task specific. For instance imitation learning for robotics may require expensive sensors. For cartpole and similar RL environments have a look at <a href="https://github.com/HumanCompatibleAI/imitation" rel="nofollow noreferrer">https://github.com/HumanCompatibleAI/imitation</a>, they have scripts to learn from RL agents.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-04T11:22:08.180Z
Answers.Text: @Rexcius I was wondering if humans could do that, because if taks is for robotic arm to pick and place an object, then humans might do it with 2 different trajectory, and when those trajectories get into training set dnn would be `confused` because it will not know which path to optimize for (slight deviation is okay but not the huge ones)
Answers.UserDisplayName: dato nefaridze
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-04T13:59:22.180Z
Answers.Text: That is why I said it's hard. The easier path for development is to have the humans use the same interface of the robots, for instance teleoperating a robot arm with a joystick, and using the robotic joint positions and velocities as data. But this may result in suboptimal demonstrations, since humans do not optimise how a robot could move.
Answers.UserDisplayName: Rexcirus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-04T14:02:05.577Z
Answers.Text: Regarding having multiple trajectories for the same goal, that is fine, it actually makes your dataset richer. Of course you need to use smart learning strategies than simply averaging trajectories. But that is why you should use existing imitation learning libraries.
Answers.UserDisplayName: Rexcirus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-04T14:21:40.403Z
Answers.Text: no it's not easy, imagine you picked cube from left trajectory from once, and then you did the same but with right trajectory, now robot is reset, in which way should it go? one time you told it that it should go left on the given state and other time you told it to go right on the given state
Answers.UserDisplayName: dato nefaridze
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-04T14:39:22.180Z
Answers.Text: It's up to you  to either provide more demonstration or providing only coherent demonstrations (e.g. only pick cube from the left). The first option is more general.
Answers.UserDisplayName: Rexcirus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-03T21:00:40.427Z
Answers.LastActivityDate: 2023-01-03T21:00:40.427Z
Answers.UserDisplayName: Rexcirus
Body: <p>How can imitation learning data be collected? Can I use a neural network for that? It might be noisy. Should I use manual gathering?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-03T20:25:45.540Z
Comments.Text: Imitation data of what process?
Comments.UserDisplayName: Rexcirus
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-03T20:42:07.130Z
Comments.Text: @Rexcirus what do you mean? suppose i am training dnn, on cartpole, how can i collect data on cartpole?
Comments.UserDisplayName: dato nefaridze
Tags: <reinforcement-learning><deep-learning><datasets><imitation-learning><data-collection>
Title: How can imitation learning data be collected?
UserDisplayName: dato nefaridze
ThreadText: # How can imitation learning data be collected?
  
  (Asked by: username_0 on 2023-01-03)
  
  How can imitation learning data be collected? Can I use a neural network for that? It might be noisy. Should I use manual gathering?
  
  **Tags:** reinforcement-learning, deep-learning, datasets, imitation-learning, data-collection
  > **Comments:**
  > **username_1**: Imitation data of what process?  
  > **username_0**: @Rexcirus what do you mean? suppose i am training dnn, on cartpole, how can i collect data on cartpole?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-01-03. Score: 1
  
  Imitation learning data usually means data gathered from an expert, that is data from an agent proficient in the task.
  
  The agent may be:
  
  * A human operator: have the operator complete the task and record inputs and action taken.
  * A pre-trained reinforcement learning agent. Same as above.
  
  The collected data is exactly the same for both cases.
  
  The hard part is usually building an interface to collect this data, which is very task specific. For instance imitation learning for robotics may require expensive sensors. For cartpole and similar RL environments have a look at <https://github.com/HumanCompatibleAI/imitation>, they have scripts to learn from RL agents.
  
  > **Comments:**
  > **username_0**: @Rexcius I was wondering if humans could do that, because if taks is for robotic arm to pick and place an object, then humans might do it with 2 different trajectory, and when those trajectories get into training set dnn would be `confused` because it will not know which path to optimize for (slight deviation is okay but not the huge ones)  
  > **username_1**: That is why I said it's hard. The easier path for development is to have the humans use the same interface of the robots, for instance teleoperating a robot arm with a joystick, and using the robotic joint positions and velocities as data. But this may result in suboptimal demonstrations, since humans do not optimise how a robot could move.  
  > **username_1**: Regarding having multiple trajectories for the same goal, that is fine, it actually makes your dataset richer. Of course you need to use smart learning strategies than simply averaging trajectories. But that is why you should use existing imitation learning libraries.  
  > **username_0**: no it's not easy, imagine you picked cube from left trajectory from once, and then you did the same but with right trajectory, now robot is reset, in which way should it go? one time you told it that it should go left on the given state and other time you told it to go right on the given state  
  > **username_1**: It's up to you  to either provide more demonstration or providing only coherent demonstrations (e.g. only pick cube from the left). The first option is more general.  
  
  ---
  
  