RobustDexGrasp: Robust Dexterous Grasping of General Objects from
  Single-view Perception
Robust grasping of various objects from single-view perception is fundamental
for dexterous robots. Previous works often rely on fully observable objects,
expert demonstrations, or static grasping poses, which restrict their
generalization ability and adaptability to external disturbances. In this
paper, we present a reinforcement-learning-based framework that enables
zero-shot dynamic dexterous grasping of a wide range of unseen objects from
single-view perception, while performing adaptive motions to external
disturbances. We utilize a hand-centric object representation for shape feature
extraction that emphasizes interaction-relevant local shapes, enhancing
robustness to shape variance and uncertainty. To enable effective hand
adaptation to disturbances with limited observations, we propose a mixed
curriculum learning strategy, which first utilizes imitation learning to
distill a policy trained with privileged real-time visual-tactile feedback, and
gradually transfers to reinforcement learning to learn adaptive motions under
disturbances caused by observation noises and dynamic randomization. Our
experiments demonstrate strong generalization in grasping unseen objects with
random poses, achieving success rates of 97.0% across 247,786 simulated objects
and 94.6% across 512 real objects. We also demonstrate the robustness of our
method to various disturbances, including unobserved object movement and
external forces, through both quantitative and qualitative evaluations. Project
Page: https://zdchan.github.io/Robust_DexGrasp/
67f6b394b67801f1ab494709
Hui Zhang
claimed_verified
2025-04-10T06:45:15.453Z
67f6b0fd2142abc30f1a193e
/avatars/abc81a1bef1055da378c780d435dcc0a.svg
Hui Zhang
user
ethHuiZhang
67f6b394b67801f1ab49470a
Zijian Wu
67f6b394b67801f1ab49470b
Linyi Huang
67f6b394b67801f1ab49470c
Sammy Christen
67f6b394b67801f1ab49470d
Jie Song
RobustDexGrasp: Robust Dexterous Grasping of General Objects from
  Single-view Perception
Robust grasping of various objects from single-view perception is fundamental
for dexterous robots. Previous works often rely on fully observable objects,
expert demonstrations, or static grasping poses, which restrict their
generalization ability and adaptability to external disturbances. In this
paper, we present a reinforcement-learning-based framework that enables
zero-shot dynamic dexterous grasping of a wide range of unseen objects from
single-view perception, while performing adaptive motions to external
disturbances. We utilize a hand-centric object representation for shape feature
extraction that emphasizes interaction-relevant local shapes, enhancing
robustness to shape variance and uncertainty. To enable effective hand
adaptation to disturbances with limited observations, we propose a mixed
curriculum learning strategy, which first utilizes imitation learning to
distill a policy trained with privileged real-time visual-tactile feedback, and
gradually transfers to reinforcement learning to learn adaptive motions under
disturbances caused by observation noises and dynamic randomization. Our
experiments demonstrate strong generalization in grasping unseen objects with
random poses, achieving success rates of 97.0% across 247,786 simulated objects
and 94.6% across 512 real objects. We also demonstrate the robustness of our
method to various disturbances, including unobserved object movement and
external forces, through both quantitative and qualitative evaluations. Project
Page: https://zdchan.github.io/Robust_DexGrasp/
A reinforcement-learning-based framework enables dynamic and robust grasping from single-view perception, adapting to disturbances and generalizing to unseen objects.
reinforcement-learning-based framework zero-shot dynamic grasping single-view perception adaptive motions hand-centric object representation mixed curriculum learning imitation learning reinforcement learning privileged real-time visual-tactile feedback dynamic randomization grasping success rates simulated objects real objects robustness to disturbances unobserved object movement external forces