title: Lessons from Learning to Spin "Pens"
summary: In-hand manipulation of pen-like objects is an important skill in our daily
  lives, as many tools such as hammers and screwdrivers are similarly shaped.
  However, current learning-based methods struggle with this task due to a lack
  of high-quality demonstrations and the significant gap between simulation and
  the real world. In this work, we push the boundaries of learning-based in-hand
  manipulation systems by demonstrating the capability to spin pen-like objects.
  We first use reinforcement learning to train an oracle policy with privileged
  information and generate a high-fidelity trajectory dataset in simulation. This
  serves two purposes: 1) pre-training a sensorimotor policy in simulation; 2)
  conducting open-loop trajectory replay in the real world. We then fine-tune the
  sensorimotor policy using these real-world trajectories to adapt it to the real
  world dynamics. With less than 50 trajectories, our policy learns to rotate
  more than ten pen-like objects with different physical properties for multiple
  revolutions. We present a comprehensive analysis of our design choices and
  share the lessons learned during development.
paper_authors._id: 66a701400651bc96877907f9
paper_authors.name: Jun Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-29T14:58:05.196Z
paper_authors._id: 6406fdc42e309e65451deb6b
paper_authors.avatarUrl: /avatars/c94326a5746330a3ad050b9c4191f4eb.svg
paper_authors.fullname: Jun Wang
paper_authors.type: user
paper_authors.user: Junw048
paper_authors._id: 66a701400651bc96877907fa
paper_authors.name: Ying Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-29T14:58:06.827Z
paper_authors._id: 66a78bbad794ddcd151441fb
paper_authors.avatarUrl: /avatars/210f139d0c40d4c051deb46a4477d073.svg
paper_authors.fullname: Ying Yuan
paper_authors.type: user
paper_authors.user: YingYuan0414
paper_authors._id: 66a701400651bc96877907fb
paper_authors.name: Haichuan Che
paper_authors._id: 66a701400651bc96877907fc
paper_authors.name: Haozhi Qi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-29T08:52:07.920Z
paper_authors._id: 658871e7d1331d552be8b457
paper_authors.avatarUrl: /avatars/c323178ad1e0fbdfa5e66dcd4ba357e6.svg
paper_authors.fullname: Haozhi Qi
paper_authors.type: user
paper_authors.user: HaozhiQi
paper_authors._id: 66a701400651bc96877907fd
paper_authors.name: Yi Ma
paper_authors._id: 66a701400651bc96877907fe
paper_authors.name: Jitendra Malik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-29T08:52:24.489Z
paper_authors._id: 65369a95605a07338de78ab0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sGFjOjLT2akN-sn5beVWL.jpeg
paper_authors.fullname: Jitendra Malik 
paper_authors.type: user
paper_authors.user: jitendra1995
paper_authors._id: 66a701400651bc96877907ff
paper_authors.name: Xiaolong Wang
paper_title: Lessons from Learning to Spin "Pens"
paper_summary: In-hand manipulation of pen-like objects is an important skill in our daily
  lives, as many tools such as hammers and screwdrivers are similarly shaped.
  However, current learning-based methods struggle with this task due to a lack
  of high-quality demonstrations and the significant gap between simulation and
  the real world. In this work, we push the boundaries of learning-based in-hand
  manipulation systems by demonstrating the capability to spin pen-like objects.
  We first use reinforcement learning to train an oracle policy with privileged
  information and generate a high-fidelity trajectory dataset in simulation. This
  serves two purposes: 1) pre-training a sensorimotor policy in simulation; 2)
  conducting open-loop trajectory replay in the real world. We then fine-tune the
  sensorimotor policy using these real-world trajectories to adapt it to the real
  world dynamics. With less than 50 trajectories, our policy learns to rotate
  more than ten pen-like objects with different physical properties for multiple
  revolutions. We present a comprehensive analysis of our design choices and
  share the lessons learned during development.
paper_ai_keywords: reinforcement learning oracle policy high-fidelity trajectory dataset sensorimotor policy open-loop trajectory replay fine-tuning real-world dynamics