Answers.Body: <p>You can look into the techniques used in GANs (genereative adverserial networks). These networks work by having 2 learning agents. 1 to create images and 1 to learn the difference in a human made image and a computer generated image. This works because the 2 agents drive each other to be better and ultimately make the generator create images which can't be distinguished from real life images.</p>
  
  <p>In your case you can make a agent trying to tell if the data is human or computer generated. The agent learning to move will then get negative rewards when the other agent can identify it as computer movement. This way the mover will learn to move like your reference data.</p>
  
  <p>UPDATE:</p>
  
  <p>I just found this video and paper which does exactly the same as you asked. Instead of using a GAN like structure they use a task specific reward and a imitation reward, which is based on reference motion data they have.</p>
  
  <p><a href="https://www.youtube.com/watch?v=vppFvq2quQ0" rel="nofollow noreferrer">https://www.youtube.com/watch?v=vppFvq2quQ0</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-16T12:52:58.897Z
Answers.Text: To my knowledge it hasn't been done yet. Is it implementable? sure, if you look at the successes of GANs it should be able to work. to only real hard problem is getting enough reference data and figuring the right balance between moving fast and moving like a human I would guess. This is from someone who hasn't implemented a GAN before though, so I don't know how valuable my opinion on this is.
Answers.UserDisplayName: Lustwelpintje
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-16T10:23:32.463Z
Answers.LastActivityDate: 2019-11-12T14:52:22.020Z
Answers.LastEditDate: 2019-11-12T14:52:22.020Z
Answers.UserDisplayName: Lustwelpintje
Body: <p>In Reinforcement Learning, when I train a model, it comes up with its own set of solutions. For example, if I am training a robot to walk, it will come up with its own walking gait, such as <a href="https://youtu.be/hx_bgoTF7bs?t=88" rel="nofollow noreferrer">this Deep Mind robot</a> that has learned to walk in a bizarre gait. It can surely walk/run although the movements does not quite look like a human.  </p>
  
  <p>I was wondering how can I train a model by providing it some kind of reference motion data? For example, if I collect motion data from a walking human and then provide it to the training, can the training be made learn the walking movements that looks similar to the reference motion data? </p>
  
  <p>Searching online I did find some links that shows this is possible. For example, <a href="https://youtu.be/9yqa0wfw2lA?t=90" rel="nofollow noreferrer">here is a research</a> where the researchers did exactly what I am trying to do, they fed motion data captured from humans to a simulation and made it learn the movements. </p>
  
  <p>So, my question is: How can I give some kind of hints or reference data to a reinforcement learning model instead of just leaving it all by itself? how exactly is this done? What is it even called? What are some terms and keywords that I can search for to learn more? </p>
  
  <p>Many thanks in advance</p>
  
Tags: <machine-learning><deep-learning><reinforcement-learning><training><reference-request>
Title: Reinforcement learning with hints or reference model
UserDisplayName: Dmitry
ThreadText: # Reinforcement learning with hints or reference model
  
  (Asked by: username_0 on 2019-09-10)
  
  In Reinforcement Learning, when I train a model, it comes up with its own set of solutions. For example, if I am training a robot to walk, it will come up with its own walking gait, such as [this Deep Mind robot](https://youtu.be/hx_bgoTF7bs?t=88) that has learned to walk in a bizarre gait. It can surely walk/run although the movements does not quite look like a human.
  
  I was wondering how can I train a model by providing it some kind of reference motion data? For example, if I collect motion data from a walking human and then provide it to the training, can the training be made learn the walking movements that looks similar to the reference motion data?
  
  Searching online I did find some links that shows this is possible. For example, [here is a research](https://youtu.be/9yqa0wfw2lA?t=90) where the researchers did exactly what I am trying to do, they fed motion data captured from humans to a simulation and made it learn the movements.
  
  So, my question is: How can I give some kind of hints or reference data to a reinforcement learning model instead of just leaving it all by itself? how exactly is this done? What is it even called? What are some terms and keywords that I can search for to learn more?
  
  Many thanks in advance
  
  **Tags:** machine-learning, deep-learning, reinforcement-learning, training, reference-request
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-09-16. Score: 2
  
  You can look into the techniques used in GANs (genereative adverserial networks). These networks work by having 2 learning agents. 1 to create images and 1 to learn the difference in a human made image and a computer generated image. This works because the 2 agents drive each other to be better and ultimately make the generator create images which can't be distinguished from real life images.
  
  In your case you can make a agent trying to tell if the data is human or computer generated. The agent learning to move will then get negative rewards when the other agent can identify it as computer movement. This way the mover will learn to move like your reference data.
  
  UPDATE:
  
  I just found this video and paper which does exactly the same as you asked. Instead of using a GAN like structure they use a task specific reward and a imitation reward, which is based on reference motion data they have.
  
  <https://www.youtube.com/watch?v=vppFvq2quQ0>
  
  > **Comments:**
  > **username_1**: To my knowledge it hasn't been done yet. Is it implementable? sure, if you look at the successes of GANs it should be able to work. to only real hard problem is getting enough reference data and figuring the right balance between moving fast and moving like a human I would guess. This is from someone who hasn't implemented a GAN before though, so I don't know how valuable my opinion on this is.  
  
  ---
  
  