TacSL: A Library for Visuotactile Sensor Simulation and Learning
For both humans and robots, the sense of touch, known as tactile sensing, is
critical for performing contact-rich manipulation tasks. Three key challenges
in robotic tactile sensing are 1) interpreting sensor signals, 2) generating
sensor signals in novel scenarios, and 3) learning sensor-based policies. For
visuotactile sensors, interpretation has been facilitated by their close
relationship with vision sensors (e.g., RGB cameras). However, generation is
still difficult, as visuotactile sensors typically involve contact,
deformation, illumination, and imaging, all of which are expensive to simulate;
in turn, policy learning has been challenging, as simulation cannot be
leveraged for large-scale data collection. We present TacSL
(taxel), a library for GPU-based visuotactile sensor simulation and
learning. TacSL can be used to simulate visuotactile images and
extract contact-force distributions over 200times faster than the prior
state-of-the-art, all within the widely-used Isaac Gym simulator. Furthermore,
TacSL provides a learning toolkit containing multiple sensor models,
contact-intensive training environments, and online/offline algorithms that can
facilitate policy learning for sim-to-real applications. On the algorithmic
side, we introduce a novel online reinforcement-learning algorithm called
asymmetric actor-critic distillation (\sysName), designed to effectively and
efficiently learn tactile-based policies in simulation that can transfer to the
real world. Finally, we demonstrate the utility of our library and algorithms
by evaluating the benefits of distillation and multimodal sensing for
contact-rich manip ulation tasks, and most critically, performing sim-to-real
transfer. Supplementary videos and results are at
https://iakinola23.github.io/tacsl/.
66bc2863bfa218d3b155d874
Iretiayo Akinola
66bc2863bfa218d3b155d875
Jie Xu
claimed_verified
2024-08-16T21:02:53.130Z
66bf785fdccd1164803e0270
/avatars/e762628f8b8f18cefd6a8ed3a7810ec9.svg
Jie Xu
user
xujie7979
66bc2863bfa218d3b155d876
Jan Carius
66bc2863bfa218d3b155d877
Dieter Fox
66bc2863bfa218d3b155d878
Yashraj Narang
TacSL: A Library for Visuotactile Sensor Simulation and Learning
For both humans and robots, the sense of touch, known as tactile sensing, is
critical for performing contact-rich manipulation tasks. Three key challenges
in robotic tactile sensing are 1) interpreting sensor signals, 2) generating
sensor signals in novel scenarios, and 3) learning sensor-based policies. For
visuotactile sensors, interpretation has been facilitated by their close
relationship with vision sensors (e.g., RGB cameras). However, generation is
still difficult, as visuotactile sensors typically involve contact,
deformation, illumination, and imaging, all of which are expensive to simulate;
in turn, policy learning has been challenging, as simulation cannot be
leveraged for large-scale data collection. We present TacSL
(taxel), a library for GPU-based visuotactile sensor simulation and
learning. TacSL can be used to simulate visuotactile images and
extract contact-force distributions over 200times faster than the prior
state-of-the-art, all within the widely-used Isaac Gym simulator. Furthermore,
TacSL provides a learning toolkit containing multiple sensor models,
contact-intensive training environments, and online/offline algorithms that can
facilitate policy learning for sim-to-real applications. On the algorithmic
side, we introduce a novel online reinforcement-learning algorithm called
asymmetric actor-critic distillation (\sysName), designed to effectively and
efficiently learn tactile-based policies in simulation that can transfer to the
real world. Finally, we demonstrate the utility of our library and algorithms
by evaluating the benefits of distillation and multimodal sensing for
contact-rich manip ulation tasks, and most critically, performing sim-to-real
transfer. Supplementary videos and results are at
https://iakinola23.github.io/tacsl/.
TacSL is a GPU-based library for simulating and learning visuotactile sensor signals, enabling efficient simulation, policy learning, and sim-to-real transfer in contact-rich manipulation tasks.
visuotactile sensors tactile sensing sensor signals Isaac Gym asymmetric actor-critic distillation policy learning contact-force distributions contact-intensive training sim-to-real transfer reinforcement learning multimodal sensing