title: Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal
    Rearrangement
summary: We propose a system for rearranging objects in a scene to achieve a desired
  object-scene placing relationship, such as a book inserted in an open slot of a
  bookshelf. The pipeline generalizes to novel geometries, poses, and layouts of
  both scenes and objects, and is trained from demonstrations to operate directly
  on 3D point clouds. Our system overcomes challenges associated with the
  existence of many geometrically-similar rearrangement solutions for a given
  scene. By leveraging an iterative pose de-noising training procedure, we can
  fit multi-modal demonstration data and produce multi-modal outputs while
  remaining precise and accurate. We also show the advantages of conditioning on
  relevant local geometric features while ignoring irrelevant global structure
  that harms both generalization and precision. We demonstrate our approach on
  three distinct rearrangement tasks that require handling multi-modality and
  generalization over object shape and pose in both simulation and the real
  world. Project website, code, and videos:
  https://anthonysimeonov.github.io/rpdiff-multi-modal/
paper_authors._id: 64acd65c0078182535e73448
paper_authors.name: Anthony Simeonov
paper_authors._id: 64acd65c0078182535e73449
paper_authors.name: Ankit Goyal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-15T07:46:54.737Z
paper_authors._id: 666c6f67bc840e6748554a91
paper_authors.avatarUrl: /avatars/72121715e14f720e5c1d029b7f00d55d.svg
paper_authors.fullname: Ankit Goyal
paper_authors.type: user
paper_authors.user: ankgoyal
paper_authors._id: 64acd65c0078182535e7344a
paper_authors.name: Lucas Manuelli
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-12T07:29:11.525Z
paper_authors._id: 64ad76bc0d0826bdd8a32221
paper_authors.avatarUrl: /avatars/571f75f2f5710d11b2d44af8c240dd13.svg
paper_authors.fullname: Lucas Manuelli
paper_authors.type: user
paper_authors.user: lmanuelli
paper_authors._id: 64acd65c0078182535e7344b
paper_authors.name: Lin Yen-Chen
paper_authors._id: 64acd65c0078182535e7344c
paper_authors.name: Alina Sarmiento
paper_authors._id: 64acd65c0078182535e7344d
paper_authors.name: Alberto Rodriguez
paper_authors._id: 64acd65c0078182535e7344e
paper_authors.name: Pulkit Agrawal
paper_authors._id: 64acd65c0078182535e7344f
paper_authors.name: Dieter Fox
paper_title: Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal
    Rearrangement
paper_summary: We propose a system for rearranging objects in a scene to achieve a desired
  object-scene placing relationship, such as a book inserted in an open slot of a
  bookshelf. The pipeline generalizes to novel geometries, poses, and layouts of
  both scenes and objects, and is trained from demonstrations to operate directly
  on 3D point clouds. Our system overcomes challenges associated with the
  existence of many geometrically-similar rearrangement solutions for a given
  scene. By leveraging an iterative pose de-noising training procedure, we can
  fit multi-modal demonstration data and produce multi-modal outputs while
  remaining precise and accurate. We also show the advantages of conditioning on
  relevant local geometric features while ignoring irrelevant global structure
  that harms both generalization and precision. We demonstrate our approach on
  three distinct rearrangement tasks that require handling multi-modality and
  generalization over object shape and pose in both simulation and the real
  world. Project website, code, and videos:
  https://anthonysimeonov.github.io/rpdiff-multi-modal/
paper_ai_keywords: iterative pose de-noising training procedure multi-modal demonstration data multi-modal outputs confidence textual textual low-dimensional geometric features generalization over object shape and pose