PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation
<|reference_start|>PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation: Humans seemingly incorporate potential touch signals in their perception. Our goal is to equip robots with a similar capability, which we term \ourmodel. \ourmodel aims to predict the expected touch signal based on a visual patch representing the touched area. We frame this problem as the task of learning a low-dimensional visual-tactile embedding, wherein we encode a depth patch from which we decode the tactile signal. To accomplish this task, we employ ReSkin, an inexpensive and replaceable magnetic-based tactile sensor. Using ReSkin, we collect and train PseudoTouch on a dataset comprising aligned tactile and visual data pairs obtained through random touching of eight basic geometric shapes. We demonstrate the efficacy of PseudoTouch through its application to two downstream tasks: object recognition and grasp stability prediction. In the object recognition task, we evaluate the learned embedding's performance on a set of five basic geometric shapes and five household objects. Using PseudoTouch, we achieve an object recognition accuracy 84% after just ten touches, surpassing a proprioception baseline. For the grasp stability task, we use ACRONYM labels to train and evaluate a grasp success predictor using PseudoTouch's predictions derived from virtual depth information. Our approach yields an impressive 32% absolute improvement in accuracy compared to the baseline relying on partial point cloud data. We make the data, code, and trained models publicly available at http://pseudotouch.cs.uni-freiburg.de.<|reference_end|>
@article{röfer2024pseudotouch:,
    title={PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic
  Manipulation},
    author={Adrian R"ofer, Nick Heppert, Abdallah Ayman, Eugenio Chisari, Abhinav
  Valada},
    journal={arXiv preprint arXiv:2403.15107},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.15107},
    primaryClass={cs.RO cs.CV}
}