title: pix2gestalt: Amodal Segmentation by Synthesizing Wholes
summary: We introduce pix2gestalt, a framework for zero-shot amodal segmentation,
  which learns to estimate the shape and appearance of whole objects that are
  only partially visible behind occlusions. By capitalizing on large-scale
  diffusion models and transferring their representations to this task, we learn
  a conditional diffusion model for reconstructing whole objects in challenging
  zero-shot cases, including examples that break natural and physical priors,
  such as art. As training data, we use a synthetically curated dataset
  containing occluded objects paired with their whole counterparts. Experiments
  show that our approach outperforms supervised baselines on established
  benchmarks. Our model can furthermore be used to significantly improve the
  performance of existing object recognition and 3D reconstruction methods in the
  presence of occlusions.
paper_authors._id: 65b33a5a2be9660f0b425af0
paper_authors.name: Ege Ozguroglu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-26T08:23:36.337Z
paper_authors._id: 65b34a907cd0069ad64b5daa
paper_authors.avatarUrl: /avatars/efc321bd987ad907c53c5a07e81a6aea.svg
paper_authors.fullname: Ege Ozguroglu
paper_authors.type: user
paper_authors.user: egeozguroglu
paper_authors._id: 65b33a5a2be9660f0b425af1
paper_authors.name: Ruoshi Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:48:03.265Z
paper_authors._id: 63408322ea76a1e0b47b50e8
paper_authors.avatarUrl: /avatars/8e2453313006fdf70b99bc7d29c738b1.svg
paper_authors.fullname: Ruoshi Liu
paper_authors.type: user
paper_authors.user: rliu
paper_authors._id: 65b33a5a2be9660f0b425af2
paper_authors.name: Dídac Surís
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:48:15.447Z
paper_authors._id: 6303e8314ec2dfa82a56ed8b
paper_authors.avatarUrl: /avatars/91a80a05396ad575ca50571035b2bc09.svg
paper_authors.fullname: Didac Suris
paper_authors.type: user
paper_authors.user: didacsuris
paper_authors._id: 65b33a5a2be9660f0b425af3
paper_authors.name: Dian Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:48:37.439Z
paper_authors._id: 63a240fbe36f2e4d5b1b393c
paper_authors.avatarUrl: /avatars/2cf95ab486d3473cf0184b3844a21987.svg
paper_authors.fullname: Dian Chen
paper_authors.type: user
paper_authors.user: dianchen
paper_authors._id: 65b33a5a2be9660f0b425af4
paper_authors.name: Achal Dave
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:48:52.905Z
paper_authors._id: 6488e29b78570cedad71eb2d
paper_authors.avatarUrl: /avatars/0d8bb52c1e2be8d6aa7c732cf92fc5a9.svg
paper_authors.fullname: Achal Dave
paper_authors.type: user
paper_authors.user: achal-tri
paper_authors._id: 65b33a5a2be9660f0b425af5
paper_authors.name: Pavel Tokmakov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:48:58.780Z
paper_authors._id: 65aeab2201b6f61e24323339
paper_authors.avatarUrl: /avatars/2c119173c0871f59df108f9bb4471954.svg
paper_authors.fullname: Pavel Tokmakov
paper_authors.type: user
paper_authors.user: ptokmakov
paper_authors._id: 65b33a5a2be9660f0b425af6
paper_authors.name: Carl Vondrick
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:49:05.228Z
paper_authors._id: 63a4bfcbcde2b28f82a2c50c
paper_authors.avatarUrl: /avatars/9d978a6f0caf6ad92ec4138a84a39e53.svg
paper_authors.fullname: Carl Vondrick
paper_authors.type: user
paper_authors.user: cvondrick
paper_title: pix2gestalt: Amodal Segmentation by Synthesizing Wholes
paper_summary: We introduce pix2gestalt, a framework for zero-shot amodal segmentation,
  which learns to estimate the shape and appearance of whole objects that are
  only partially visible behind occlusions. By capitalizing on large-scale
  diffusion models and transferring their representations to this task, we learn
  a conditional diffusion model for reconstructing whole objects in challenging
  zero-shot cases, including examples that break natural and physical priors,
  such as art. As training data, we use a synthetically curated dataset
  containing occluded objects paired with their whole counterparts. Experiments
  show that our approach outperforms supervised baselines on established
  benchmarks. Our model can furthermore be used to significantly improve the
  performance of existing object recognition and 3D reconstruction methods in the
  presence of occlusions.
paper_ai_keywords: pix2gestalt zero-shot amodal segmentation conditional diffusion model diffusion models occlusions synthetically curated dataset object recognition 3D reconstruction