One-Shot Imitation under Mismatched Execution
<|reference_start|>One-Shot Imitation under Mismatched Execution: Human demonstrations as prompts are a powerful way to program robots to do long-horizon manipulation tasks. However, translating these demonstrations into robot-executable actions presents significant challenges due to execution mismatches in movement styles and physical capabilities. Existing methods either depend on robot-demonstrator paired data, which is infeasible to scale, or rely too heavily on frame-level visual similarities that often break down in practice. To address these challenges, we propose RHyME, a novel framework that automatically aligns robot and demonstrator task executions using optimal transport costs. Given long-horizon robot demonstrations, RHyME synthesizes semantically equivalent demonstrator videos by retrieving and composing short-horizon demonstrator clips. This approach facilitates effective policy training without the need for paired data. We demonstrate that RHyME outperforms a range of baselines across cross-embodiment datasets, showing a 52% increase in task recall over prior cross-embodiment learning methods. We release our code and datasets at https://portal-cornell.github.io/rhyme/.<|reference_end|>
@article{kedia2024one-shot,
    title={One-Shot Imitation under Mismatched Execution},
    author={Kushal Kedia, Prithwish Dan, Angela Chao, Maximus Adrian Pace,
  Sanjiban Choudhury},
    journal={arXiv preprint arXiv:2409.06615},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.06615},
    primaryClass={cs.RO cs.AI cs.LG}
}