UniSkill: Imitating Human Videos via Cross-Embodiment Skill
  Representations
Mimicry is a fundamental learning mechanism in humans, enabling individuals
to learn new tasks by observing and imitating experts. However, applying this
ability to robots presents significant challenges due to the inherent
differences between human and robot embodiments in both their visual appearance
and physical capabilities. While previous methods bridge this gap using
cross-embodiment datasets with shared scenes and tasks, collecting such aligned
data between humans and robots at scale is not trivial. In this paper, we
propose UniSkill, a novel framework that learns embodiment-agnostic skill
representations from large-scale cross-embodiment video data without any
labels, enabling skills extracted from human video prompts to effectively
transfer to robot policies trained only on robot data. Our experiments in both
simulation and real-world environments show that our cross-embodiment skills
successfully guide robots in selecting appropriate actions, even with unseen
video prompts. The project website can be found at:
https://kimhanjung.github.io/UniSkill.
6823f20eb9e5c4a5c866a634
Hanjung Kim
claimed_verified
2025-05-15T10:32:41.278Z
66f21f9e14fcea0aa11361f5
/avatars/fb8261fb9da98f47d9102d68762ac821.svg
Hanjung Kim
user
HanjungKim
6823f20eb9e5c4a5c866a635
Jaehyun Kang
6823f20eb9e5c4a5c866a636
Hyolim Kang
6823f20eb9e5c4a5c866a637
Meedeum Cho
6823f20eb9e5c4a5c866a638
Seon Joo Kim
6823f20eb9e5c4a5c866a639
Youngwoon Lee
UniSkill: Imitating Human Videos via Cross-Embodiment Skill
  Representations
Mimicry is a fundamental learning mechanism in humans, enabling individuals
to learn new tasks by observing and imitating experts. However, applying this
ability to robots presents significant challenges due to the inherent
differences between human and robot embodiments in both their visual appearance
and physical capabilities. While previous methods bridge this gap using
cross-embodiment datasets with shared scenes and tasks, collecting such aligned
data between humans and robots at scale is not trivial. In this paper, we
propose UniSkill, a novel framework that learns embodiment-agnostic skill
representations from large-scale cross-embodiment video data without any
labels, enabling skills extracted from human video prompts to effectively
transfer to robot policies trained only on robot data. Our experiments in both
simulation and real-world environments show that our cross-embodiment skills
successfully guide robots in selecting appropriate actions, even with unseen
video prompts. The project website can be found at:
https://kimhanjung.github.io/UniSkill.
UniSkill learns embodiment-agnostic skill representations from unlabeled cross-embodiment video data, enabling human skill transfer to robots in both simulation and real-world settings.
cross-embodiment datasets skill representations cross-embodiment skills robot policies video prompts