Low Fidelity Visuo-Tactile Pretraining Improves Vision-Only Manipulation Performance
<|reference_start|>Low Fidelity Visuo-Tactile Pretraining Improves Vision-Only Manipulation Performance: Tactile perception is a critical component of solving real-world manipulation tasks, but tactile sensors for manipulation have barriers to use such as fragility and cost. In this work, we engage a robust, low-cost tactile sensor, BeadSight, as an alternative to precise pre-calibrated sensors for a pretraining approach to manipulation. We show that tactile pretraining, even with a low-fidelity sensor as BeadSight, can improve an imitation learning agent's performance on complex manipulation tasks. We demonstrate this method against a baseline USB cable plugging task, previously achieved with a much higher precision GelSight sensor as the tactile input to pretraining. Our best BeadSight pretrained visuo-tactile agent completed the task with 70\% accuracy compared to 85\% for the best GelSight pretrained visuo-tactile agent, with vision-only inference for both.<|reference_end|>
@article{gano2024low,
    title={Low Fidelity Visuo-Tactile Pretraining Improves Vision-Only Manipulation
  Performance},
    author={Selam Gano, Abraham George, Amir Barati Farimani},
    journal={arXiv preprint arXiv:2406.15639},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.15639},
    primaryClass={cs.RO}
}