|
|
--- |
|
|
license: mit |
|
|
pipeline_tag: robotics |
|
|
--- |
|
|
|
|
|
# TokenHSI Checkpoint Instruction |
|
|
|
|
|
[2025-04-07] All models have been released! |
|
|
|
|
|
Please download the checkpoints and organize them as the following file structure: |
|
|
|
|
|
``` |
|
|
|-- tokenhsi |
|
|
|-- lpanlib |
|
|
|-- body_models |
|
|
|-- smpl |
|
|
|-- SMPL_FEMALE.npz |
|
|
|-- SMPL_MALE.npz |
|
|
|-- SMPL_NEUTRAL.pkl |
|
|
|-- ... |
|
|
|-- output |
|
|
|-- single_task |
|
|
|-- ckpt_traj.pth |
|
|
|-- ckpt_sit.pth |
|
|
|-- ckpt_climb.pth |
|
|
|-- ckpt_carry.pth |
|
|
|-- tokenhsi |
|
|
|-- ckpt_stage1.pth |
|
|
|-- ckpt_stage2_comp_traj_carry |
|
|
|-- ckpt_stage2_comp_sit_carry |
|
|
|-- ckpt_stage2_comp_climb_carry |
|
|
|-- ckpt_stage2_objShape_chair.pth |
|
|
|-- ckpt_stage2_objShape_table.pth |
|
|
|-- ckpt_stage2_terrainShape_traj.pth |
|
|
|-- ckpt_stage2_terrainShape_carry.pth |
|
|
|-- ckpt_stage2_longterm.pth |
|
|
``` |
|
|
|
|
|
* We first provide four single-task environments, including path-following, sitting, climbing, and carrying. For each task, we train a specialized policy using AMP. The trained models are: |
|
|
* single_task/ckpt_traj.pth |
|
|
* single_task/ckpt_sit.pth |
|
|
* single_task/ckpt_climb.pth |
|
|
* single_task/ckpt_carry.pth |
|
|
|
|
|
* We then provide our multi-task environment for training the versatile, unified transformer policy. The trained model is: |
|
|
* tokenhsi/ckpt_stage1.pth |
|
|
|
|
|
* Finally, we perform policy adaptation to generalize the learned skills to novel tasks. We list the trained models across various downstream tasks. |
|
|
* Skill Composition |
|
|
* Follow + Carry: tokenhsi/ckpt_stage2_comp_traj_carry.pth |
|
|
* Sit + Carry: tokenhsi/ckpt_stage2_comp_sit_carry.pth |
|
|
* Climb + Carry: tokenhsi/ckpt_stage2_comp_climb_carry.pth |
|
|
* Object Shape Variation |
|
|
* Chair: tokenhsi/ckpt_stage2_objShape_chair.pth |
|
|
* Table: tokenhsi/ckpt_stage2_objShape_table.pth |
|
|
* Terrain Shape Variation |
|
|
* Follow: tokenhsi/ckpt_stage2_terrainShape_traj.pth |
|
|
* Carry: tokenhsi/ckpt_stage2_terrainShape_carry.pth |
|
|
* Long-horizon Task Completion |
|
|
* tokenhsi/ckpt_stage2_longterm.pth |
|
|
|
|
|
GitHub repo: https://github.com/liangpan99/TokenHSI |
|
|
arXiv paper: https://huggingface.co/papers/2503.19901 |
|
|
Project page: https://liangpan99.github.io/TokenHSI/ |
