D-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous Deformable Manipulation
<|reference_start|>D-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous Deformable Manipulation: Mastering dexterous robotic manipulation of deformable objects is vital for overcoming the limitations of parallel grippers in real-world applications. Current trajectory optimisation approaches often struggle to solve such tasks due to the large search space and the limited task information available from a cost function. In this work, we propose D-Cubed, a novel trajectory optimisation method using a latent diffusion model (LDM) trained from a task-agnostic play dataset to solve dexterous deformable object manipulation tasks. D-Cubed learns a skill-latent space that encodes short-horizon actions in the play dataset using a VAE and trains a LDM to compose the skill latents into a skill trajectory, representing a long-horizon action trajectory in the dataset. To optimise a trajectory for a target task, we introduce a novel gradient-free guided sampling method that employs the Cross-Entropy method within the reverse diffusion process. In particular, D-Cubed samples a small number of noisy skill trajectories using the LDM for exploration and evaluates the trajectories in simulation. Then, D-Cubed selects the trajectory with the lowest cost for the subsequent reverse process. This effectively explores promising solution areas and optimises the sampled trajectories towards a target task throughout the reverse diffusion process. Through empirical evaluation on a public benchmark of dexterous deformable object manipulation tasks, we demonstrate that D-Cubed outperforms traditional trajectory optimisation and competitive baseline approaches by a significant margin. We further demonstrate that trajectories found by D-Cubed readily transfer to a real-world LEAP hand on a folding task.<|reference_end|>
@article{yamada2024d-cubed:,
    title={D-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous
  Deformable Manipulation},
    author={Jun Yamada, Shaohong Zhong, Jack Collins, Ingmar Posner},
    journal={arXiv preprint arXiv:2403.12861},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.12861},
    primaryClass={cs.RO cs.LG}
}