Generalizable Humanoid Manipulation with Improved 3D Diffusion Policies
Humanoid robots capable of autonomous operation in diverse environments have
long been a goal for roboticists. However, autonomous manipulation by humanoid
robots has largely been restricted to one specific scene, primarily due to the
difficulty of acquiring generalizable skills. Recent advances in 3D visuomotor
policies, such as the 3D Diffusion Policy (DP3), have shown promise in
extending these capabilities to wilder environments. However, 3D visuomotor
policies often rely on camera calibration and point-cloud segmentation, which
present challenges for deployment on mobile robots like humanoids. In this
work, we introduce the Improved 3D Diffusion Policy (iDP3), a novel 3D
visuomotor policy that eliminates these constraints by leveraging egocentric 3D
visual representations. We demonstrate that iDP3 enables a full-sized humanoid
robot to autonomously perform skills in diverse real-world scenarios, using
only data collected in the lab. Videos are available at:
https://humanoid-manipulation.github.io
670de51d1cc9c1fdf0959026
Yanjie Ze
admin_assigned
2024-10-15T16:37:10.802Z
63509bc859bfa9a85d4220aa
/avatars/ca2cc9b87f5ca5cd51606b2f9edf89d0.svg
Yanjie Ze
user
yjze
670de51d1cc9c1fdf0959027
Zixuan Chen
claimed_verified
2024-10-15T21:17:59.981Z
64a02b4c1396c67ac07798bb
/avatars/9b1b4319edbac5faeb7586a4933791d2.svg
Eric Chen
user
zxuannn
670de51d1cc9c1fdf0959028
Wenhao Wang
670de51d1cc9c1fdf0959029
Tianyi Chen
670de51d1cc9c1fdf095902a
Xialin He
admin_assigned
2024-10-15T16:38:10.895Z
6332b75df0b2af7f685810da
/avatars/078f582f555a9dee3f7ab4d155c0a65c.svg
Xialin He
user
MuLinjiu
670de51d1cc9c1fdf095902b
Ying Yuan
670de51d1cc9c1fdf095902c
Xue Bin Peng
670de51d1cc9c1fdf095902d
Jiajun Wu
admin_assigned
2024-10-15T16:37:48.920Z
647b7b8345616c3de3627863
/avatars/8196364bffed71086ba9a0a4a62027d3.svg
Jiajun Wu
user
jiajunwu
Generalizable Humanoid Manipulation with Improved 3D Diffusion Policies
Humanoid robots capable of autonomous operation in diverse environments have
long been a goal for roboticists. However, autonomous manipulation by humanoid
robots has largely been restricted to one specific scene, primarily due to the
difficulty of acquiring generalizable skills. Recent advances in 3D visuomotor
policies, such as the 3D Diffusion Policy (DP3), have shown promise in
extending these capabilities to wilder environments. However, 3D visuomotor
policies often rely on camera calibration and point-cloud segmentation, which
present challenges for deployment on mobile robots like humanoids. In this
work, we introduce the Improved 3D Diffusion Policy (iDP3), a novel 3D
visuomotor policy that eliminates these constraints by leveraging egocentric 3D
visual representations. We demonstrate that iDP3 enables a full-sized humanoid
robot to autonomously perform skills in diverse real-world scenarios, using
only data collected in the lab. Videos are available at:
https://humanoid-manipulation.github.io
An improved 3D diffusion policy enables autonomous manipulation by humanoid robots across diverse environments using egocentric visual representations.
3D Diffusion Policy (DP3) 3D visuomotor policies egocentric 3D visual representations humanoid robots autonomous manipulation diverse environments