Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network including an encoder network and decoder network and configured to receive a network input that includes sensor data characterizing a deformable object and to process the network input to generate a network output that specifies a mesh of the deformable object. Once trained, the neural network can be deployed in a robotic system for use in allowing a motion planner to issue timely commands which adjust a currently planned motion according to the mesh in order to prevent any collision between the robot and the deformable object.
ACCELERATING ROBOTIC PLANNING FOR OPERATING ON DEFORMABLE OBJECTS