DoughNet: A Visual Predictive Model for Topological Manipulation of Deformable Objects
<|reference_start|>DoughNet: A Visual Predictive Model for Topological Manipulation of Deformable Objects: Manipulation of elastoplastic objects like dough often involves topological changes such as splitting and merging. The ability to accurately predict these topological changes that a specific action might incur is critical for planning interactions with elastoplastic objects. We present DoughNet, a Transformer-based architecture for handling these challenges, consisting of two components. First, a denoising autoencoder represents deformable objects of varying topology as sets of latent codes. Second, a visual predictive model performs autoregressive set prediction to determine long-horizon geometrical deformation and topological changes purely in latent space. Given a partial initial state and desired manipulation trajectories, it infers all resulting object geometries and topologies at each step. DoughNet thereby allows to plan robotic manipulation; selecting a suited tool, its pose and opening width to recreate robot- or human-made goals. Our experiments in simulated and real environments show that DoughNet is able to significantly outperform related approaches that consider deformation only as geometrical change.<|reference_end|>
@article{bauer2024doughnet:,
    title={DoughNet: A Visual Predictive Model for Topological Manipulation of
  Deformable Objects},
    author={Dominik Bauer, Zhenjia Xu, Shuran Song},
    journal={arXiv preprint arXiv:2404.12524},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.12524},
    primaryClass={cs.CV cs.LG cs.RO}
}