title: UniT: Unified Tactile Representation for Robot Learning
summary: UniT is a novel approach to tactile representation learning, using VQVAE to
  learn a compact latent space and serve as the tactile representation. It uses
  tactile images obtained from a single simple object to train the representation
  with transferability and generalizability. This tactile representation can be
  zero-shot transferred to various downstream tasks, including perception tasks
  and manipulation policy learning. Our benchmarking on an in-hand 3D pose
  estimation task shows that UniT outperforms existing visual and tactile
  representation learning methods. Additionally, UniT's effectiveness in policy
  learning is demonstrated across three real-world tasks involving diverse
  manipulated objects and complex robot-object-environment interactions. Through
  extensive experimentation, UniT is shown to be a simple-to-train,
  plug-and-play, yet widely effective method for tactile representation learning.
  For more details, please refer to our open-source repository
  https://github.com/ZhengtongXu/UniT and the project website
  https://zhengtongxu.github.io/unifiedtactile.github.io/.
paper_authors._id: 66bc26c5f36eed1977865083
paper_authors.name: Zhengtong Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-15T10:11:53.296Z
paper_authors._id: 66bcfb28dac7d713269cbfd2
paper_authors.avatarUrl: /avatars/1b20fefdd0cf5a7ceda515b361d56016.svg
paper_authors.fullname: Zhengtong Xu
paper_authors.type: user
paper_authors.user: Zhengtongxu
paper_authors._id: 66bc26c5f36eed1977865084
paper_authors.name: Raghava Uppuluri
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-14T11:06:37.737Z
paper_authors._id: 60b25904295203cd10efe5c1
paper_authors.avatarUrl: /avatars/d8062f3dabed336b530c7c05a61e2868.svg
paper_authors.fullname: Raghava Uppuluri
paper_authors.type: user
paper_authors.user: raghavauppuluri
paper_authors._id: 66bc26c5f36eed1977865085
paper_authors.name: Xinwei Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-15T10:11:41.534Z
paper_authors._id: 646fdb33cc049b49e35ff10c
paper_authors.avatarUrl: /avatars/53606e9b66e721a70be4b392a2ce19b3.svg
paper_authors.fullname: Xinwei Zhang
paper_authors.type: user
paper_authors.user: Cyruzzz
paper_authors._id: 66bc26c5f36eed1977865086
paper_authors.name: Cael Fitch
paper_authors._id: 66bc26c5f36eed1977865087
paper_authors.name: Philip Glen Crandall
paper_authors._id: 66bc26c5f36eed1977865088
paper_authors.name: Wan Shou
paper_authors._id: 66bc26c5f36eed1977865089
paper_authors.name: Dongyi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-14T11:07:26.310Z
paper_authors._id: 6660dc7bc629e0e089e67468
paper_authors.avatarUrl: /avatars/6878288c695bece4bb75c02bb85c7e49.svg
paper_authors.fullname: Dongyi Wang
paper_authors.type: user
paper_authors.user: DongyiWang
paper_authors._id: 66bc26c5f36eed197786508a
paper_authors.name: Yu She
paper_title: UniT: Unified Tactile Representation for Robot Learning
paper_summary: UniT is a novel approach to tactile representation learning, using VQVAE to
  learn a compact latent space and serve as the tactile representation. It uses
  tactile images obtained from a single simple object to train the representation
  with transferability and generalizability. This tactile representation can be
  zero-shot transferred to various downstream tasks, including perception tasks
  and manipulation policy learning. Our benchmarking on an in-hand 3D pose
  estimation task shows that UniT outperforms existing visual and tactile
  representation learning methods. Additionally, UniT's effectiveness in policy
  learning is demonstrated across three real-world tasks involving diverse
  manipulated objects and complex robot-object-environment interactions. Through
  extensive experimentation, UniT is shown to be a simple-to-train,
  plug-and-play, yet widely effective method for tactile representation learning.
  For more details, please refer to our open-source repository
  https://github.com/ZhengtongXu/UniT and the project website
  https://zhengtongxu.github.io/unifiedtactile.github.io/.
paper_ai_keywords: VQVAE tactile representation learning compact latent space zero-shot transfer downstream tasks perception tasks manipulation policy learning in-hand 3D pose estimation task plug-and-play