title: Grasping Diverse Objects with Simulated Humanoids
summary: We present a method for controlling a simulated humanoid to grasp an object
  and move it to follow an object trajectory. Due to the challenges in
  controlling a humanoid with dexterous hands, prior methods often use a
  disembodied hand and only consider vertical lifts or short trajectories. This
  limited scope hampers their applicability for object manipulation required for
  animation and simulation. To close this gap, we learn a controller that can
  pick up a large number (>1200) of objects and carry them to follow randomly
  generated trajectories. Our key insight is to leverage a humanoid motion
  representation that provides human-like motor skills and significantly speeds
  up training. Using only simplistic reward, state, and object representations,
  our method shows favorable scalability on diverse object and trajectories. For
  training, we do not need dataset of paired full-body motion and object
  trajectories. At test time, we only require the object mesh and desired
  trajectories for grasping and transporting. To demonstrate the capabilities of
  our method, we show state-of-the-art success rates in following object
  trajectories and generalizing to unseen objects. Code and models will be
  released.
paper_authors._id: 669735ac84248e122c79ce65
paper_authors.name: Zhengyi Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-23T07:48:27.014Z
paper_authors._id: 645eaf37f0f92653b9f2ba77
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VM-hXdknuRfeK3BD9FbyR.png
paper_authors.fullname: Luo
paper_authors.type: user
paper_authors.user: Zhengyiluo
paper_authors._id: 669735ac84248e122c79ce66
paper_authors.name: Jinkun Cao
paper_authors._id: 669735ac84248e122c79ce67
paper_authors.name: Sammy Christen
paper_authors._id: 669735ac84248e122c79ce68
paper_authors.name: Alexander Winkler
paper_authors._id: 669735ac84248e122c79ce69
paper_authors.name: Kris Kitani
paper_authors._id: 669735ac84248e122c79ce6a
paper_authors.name: Weipeng Xu
paper_title: Grasping Diverse Objects with Simulated Humanoids
paper_summary: We present a method for controlling a simulated humanoid to grasp an object
  and move it to follow an object trajectory. Due to the challenges in
  controlling a humanoid with dexterous hands, prior methods often use a
  disembodied hand and only consider vertical lifts or short trajectories. This
  limited scope hampers their applicability for object manipulation required for
  animation and simulation. To close this gap, we learn a controller that can
  pick up a large number (>1200) of objects and carry them to follow randomly
  generated trajectories. Our key insight is to leverage a humanoid motion
  representation that provides human-like motor skills and significantly speeds
  up training. Using only simplistic reward, state, and object representations,
  our method shows favorable scalability on diverse object and trajectories. For
  training, we do not need dataset of paired full-body motion and object
  trajectories. At test time, we only require the object mesh and desired
  trajectories for grasping and transporting. To demonstrate the capabilities of
  our method, we show state-of-the-art success rates in following object
  trajectories and generalizing to unseen objects. Code and models will be
  released.
paper_ai_keywords: humanoid grasp object manipulation dexterous hands embodied hand vertical lifts object trajectory controller humanoid motion representation motor skills training reward state representation object representation full-body motion object mesh grasping transporting state-of-the-art success rates