title: RobustDexGrasp: Robust Dexterous Grasping of General Objects from
    Single-view Perception
summary: Robust grasping of various objects from single-view perception is fundamental
  for dexterous robots. Previous works often rely on fully observable objects,
  expert demonstrations, or static grasping poses, which restrict their
  generalization ability and adaptability to external disturbances. In this
  paper, we present a reinforcement-learning-based framework that enables
  zero-shot dynamic dexterous grasping of a wide range of unseen objects from
  single-view perception, while performing adaptive motions to external
  disturbances. We utilize a hand-centric object representation for shape feature
  extraction that emphasizes interaction-relevant local shapes, enhancing
  robustness to shape variance and uncertainty. To enable effective hand
  adaptation to disturbances with limited observations, we propose a mixed
  curriculum learning strategy, which first utilizes imitation learning to
  distill a policy trained with privileged real-time visual-tactile feedback, and
  gradually transfers to reinforcement learning to learn adaptive motions under
  disturbances caused by observation noises and dynamic randomization. Our
  experiments demonstrate strong generalization in grasping unseen objects with
  random poses, achieving success rates of 97.0% across 247,786 simulated objects
  and 94.6% across 512 real objects. We also demonstrate the robustness of our
  method to various disturbances, including unobserved object movement and
  external forces, through both quantitative and qualitative evaluations. Project
  Page: https://zdchan.github.io/Robust_DexGrasp/
paper_authors._id: 67f6b394b67801f1ab494709
paper_authors.name: Hui Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-10T06:45:15.453Z
paper_authors._id: 67f6b0fd2142abc30f1a193e
paper_authors.avatarUrl: /avatars/abc81a1bef1055da378c780d435dcc0a.svg
paper_authors.fullname: Hui Zhang
paper_authors.type: user
paper_authors.user: ethHuiZhang
paper_authors._id: 67f6b394b67801f1ab49470a
paper_authors.name: Zijian Wu
paper_authors._id: 67f6b394b67801f1ab49470b
paper_authors.name: Linyi Huang
paper_authors._id: 67f6b394b67801f1ab49470c
paper_authors.name: Sammy Christen
paper_authors._id: 67f6b394b67801f1ab49470d
paper_authors.name: Jie Song
paper_title: RobustDexGrasp: Robust Dexterous Grasping of General Objects from
    Single-view Perception
paper_summary: Robust grasping of various objects from single-view perception is fundamental
  for dexterous robots. Previous works often rely on fully observable objects,
  expert demonstrations, or static grasping poses, which restrict their
  generalization ability and adaptability to external disturbances. In this
  paper, we present a reinforcement-learning-based framework that enables
  zero-shot dynamic dexterous grasping of a wide range of unseen objects from
  single-view perception, while performing adaptive motions to external
  disturbances. We utilize a hand-centric object representation for shape feature
  extraction that emphasizes interaction-relevant local shapes, enhancing
  robustness to shape variance and uncertainty. To enable effective hand
  adaptation to disturbances with limited observations, we propose a mixed
  curriculum learning strategy, which first utilizes imitation learning to
  distill a policy trained with privileged real-time visual-tactile feedback, and
  gradually transfers to reinforcement learning to learn adaptive motions under
  disturbances caused by observation noises and dynamic randomization. Our
  experiments demonstrate strong generalization in grasping unseen objects with
  random poses, achieving success rates of 97.0% across 247,786 simulated objects
  and 94.6% across 512 real objects. We also demonstrate the robustness of our
  method to various disturbances, including unobserved object movement and
  external forces, through both quantitative and qualitative evaluations. Project
  Page: https://zdchan.github.io/Robust_DexGrasp/
paper_ai_summary: A reinforcement-learning-based framework enables dynamic and robust grasping from single-view perception, adapting to disturbances and generalizing to unseen objects.
paper_ai_keywords: reinforcement-learning-based framework zero-shot dynamic grasping single-view perception adaptive motions hand-centric object representation mixed curriculum learning imitation learning reinforcement learning privileged real-time visual-tactile feedback dynamic randomization grasping success rates simulated objects real objects robustness to disturbances unobserved object movement external forces