Answers.Body: <p>DeepMind used <a href="http://www.mujoco.org/" rel="nofollow noreferrer">MuJoCo</a> (see also the related paper <a href="https://homes.cs.washington.edu/~todorov/papers/TodorovIROS12.pdf" rel="nofollow noreferrer">MuJoCo: A physics engine for model-based control</a>) for the simulations, as they stated in section 3.1 of their paper <a href="https://arxiv.org/pdf/1707.02286.pdf" rel="nofollow noreferrer">Emergence of Locomotion Behaviours in Rich Environments</a> (2017), which is the paper you should read to know more about their results related to those animations of skeletons that try to walk or jump (but do it weirdly).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-16T12:32:26.440Z
Answers.Text: This makes sense. I was told that Gazebo is one of the most used simulators in robotics as it has numerous builtin plugins for handing sensor data, noise. I am just confused why DeepMind and OpenAI and others develop their own version when there is something that is lot more versatile already exists. Gazebo even have solver for wind dynamics. I shall ready the paper you added to see key advantage of MuJoCo
Answers.UserDisplayName: gfdsal
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-16T12:46:19.227Z
Answers.Text: @gfdsal Honestly, I don't know right now. I could investigate that later, though.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-15T21:17:20.503Z
Answers.LastActivityDate: 2020-04-15T21:17:20.503Z
Answers.UserDisplayName: nbro
Body: <p>Which simulation platform is used by <a href="https://www.youtube.com/watch?v=Xmw3UwP_vnc" rel="nofollow noreferrer">DeepMind</a> and <a href="https://www.youtube.com/watch?v=pgaEE27nsQw" rel="nofollow noreferrer">others</a> to handle inverse kinematics musculoskeletal simulation, etc., for reinforcement learning simulations and agents?</p>
  
  <p>I thought they use Unity or Unreal but I assume that would be resource-heavy. </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-14T15:43:04.183Z
Comments.Text: I don't think there's one answer. For example, OpenAI developed their [Gym](https://gym.openai.com/) for this purpose.
Comments.UserDisplayName: Andy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-14T16:15:54.080Z
Comments.Text: @Andy makes sense. So are these all custom developed? I wished someone could have had done some study of pro's and con's for these such as Gazebo, Cinder etc. I am quite eager to adapt the platform that is lightweight (not too many libraries) and highly controlable instead of having to deal with a framework.
Comments.UserDisplayName: gfdsal
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-14T16:52:58.567Z
Comments.Text: I'm not sure, sorry, and a quick search didn't turn up much. What you're looking for is a survey of reinforcement learning toolkits, I think.
Comments.UserDisplayName: Andy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-14T22:17:22.600Z
Comments.Text: Welcome to SE:AI!
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-15T21:21:09.673Z
Comments.Text: @gfdsal You could try to ask this as a new question on our site. For example, you could ask "what are the pros and cons of simulators, such as Gazebo, Cinder, etc., for reinforcement learning tasks?" or something like that. Try to be specific and clear!
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-16T12:29:37.490Z
Comments.Text: @nbro, I shall be specific and clear in my next question.
Comments.UserDisplayName: gfdsal
Tags: <reinforcement-learning><reference-request><deepmind>
Title: Which simulation platform is used by DeepMind (and others) to handle inverse kinematics musculoskeletal?
UserDisplayName: gfdsal
ThreadText: # Which simulation platform is used by DeepMind (and others) to handle inverse kinematics musculoskeletal?
  
  (Asked by: username_0 on 2020-04-14)
  
  Which simulation platform is used by [DeepMind](https://www.youtube.com/watch?v=Xmw3UwP_vnc) and [others](https://www.youtube.com/watch?v=pgaEE27nsQw) to handle inverse kinematics musculoskeletal simulation, etc., for reinforcement learning simulations and agents?
  
  I thought they use Unity or Unreal but I assume that would be resource-heavy.
  
  **Tags:** reinforcement-learning, reference-request, deepmind
  
  > **Comments:**
  > **username_1**: I don't think there's one answer. For example, OpenAI developed their [Gym](https://gym.openai.com/) for this purpose.  
  > **username_0**: @username_1 makes sense. So are these all custom developed? I wished someone could have had done some study of pro's and con's for these such as Gazebo, Cinder etc. I am quite eager to adapt the platform that is lightweight (not too many libraries) and highly controlable instead of having to deal with a framework.  
  > **username_1**: I'm not sure, sorry, and a quick search didn't turn up much. What you're looking for is a survey of reinforcement learning toolkits, I think.  
  > **username_2**: Welcome to SE:AI!  
  > **username_3**: @username_0 You could try to ask this as a new question on our site. For example, you could ask "what are the pros and cons of simulators, such as Gazebo, Cinder, etc., for reinforcement learning tasks?" or something like that. Try to be specific and clear!  
  > **username_0**: @username_3, I shall be specific and clear in my next question.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2020-04-15. Score: 1 | Accepted answer
  
  DeepMind used [MuJoCo](http://www.mujoco.org/) (see also the related paper [MuJoCo: A physics engine for model-based control](https://homes.cs.washington.edu/~todorov/papers/TodorovIROS12.pdf)) for the simulations, as they stated in section 3.1 of their paper [Emergence of Locomotion Behaviours in Rich Environments](https://arxiv.org/pdf/1707.02286.pdf) (2017), which is the paper you should read to know more about their results related to those animations of skeletons that try to walk or jump (but do it weirdly).
  
  > **Comments:**
  > **username_0**: This makes sense. I was told that Gazebo is one of the most used simulators in robotics as it has numerous builtin plugins for handing sensor data, noise. I am just confused why DeepMind and OpenAI and others develop their own version when there is something that is lot more versatile already exists. Gazebo even have solver for wind dynamics. I shall ready the paper you added to see key advantage of MuJoCo  
  > **username_3**: @username_0 Honestly, I don't know right now. I could investigate that later, though.  
  
  ---
  
  