title: Point-Cloud Completion with Pretrained Text-to-image Diffusion Models
summary: Point-cloud data collected in real-world applications are often incomplete.
  Data is typically missing due to objects being observed from partial
  viewpoints, which only capture a specific perspective or angle. Additionally,
  data can be incomplete due to occlusion and low-resolution sampling. Existing
  completion approaches rely on datasets of predefined objects to guide the
  completion of noisy and incomplete, point clouds. However, these approaches
  perform poorly when tested on Out-Of-Distribution (OOD) objects, that are
  poorly represented in the training dataset. Here we leverage recent advances in
  text-guided image generation, which lead to major breakthroughs in text-guided
  shape generation. We describe an approach called SDS-Complete that uses a
  pre-trained text-to-image diffusion model and leverages the text semantics of a
  given incomplete point cloud of an object, to obtain a complete surface
  representation. SDS-Complete can complete a variety of objects using test-time
  optimization without expensive collection of 3D information. We evaluate SDS
  Complete on incomplete scanned objects, captured by real-world depth sensors
  and LiDAR scanners. We find that it effectively reconstructs objects that are
  absent from common datasets, reducing Chamfer loss by 50% on average compared
  with current methods. Project page: https://sds-complete.github.io/
paper_authors._id: 64927091b4ab18f4467522bf
paper_authors.name: Yoni Kasten
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-21T15:50:02.284Z
paper_authors._id: 642294c112b4b51aae368d30
paper_authors.avatarUrl: /avatars/017c19910ba01d6bd9cd864132652448.svg
paper_authors.fullname: Yoni Kasten
paper_authors.type: user
paper_authors.user: ykasten
paper_authors._id: 64927091b4ab18f4467522c0
paper_authors.name: Ohad Rahamim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-01T20:15:14.988Z
paper_authors._id: 63c59c3a6d132b995fedface
paper_authors.avatarUrl: /avatars/4e18b19e477cb683ce1ba3ae6ab77d8e.svg
paper_authors.fullname: Ohad rahamim
paper_authors.type: user
paper_authors.user: ohad204
paper_authors._id: 64927091b4ab18f4467522c1
paper_authors.name: Gal Chechik
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-21T20:46:26.908Z
paper_authors._id: 6493393f357b252af72196c5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6493393f357b252af72196c5/EWSy18XRcMRa_4XMM3Fu-.jpeg
paper_authors.fullname: Gal Chechik
paper_authors.type: user
paper_authors.user: galchechik
paper_title: Point-Cloud Completion with Pretrained Text-to-image Diffusion Models
paper_summary: Point-cloud data collected in real-world applications are often incomplete.
  Data is typically missing due to objects being observed from partial
  viewpoints, which only capture a specific perspective or angle. Additionally,
  data can be incomplete due to occlusion and low-resolution sampling. Existing
  completion approaches rely on datasets of predefined objects to guide the
  completion of noisy and incomplete, point clouds. However, these approaches
  perform poorly when tested on Out-Of-Distribution (OOD) objects, that are
  poorly represented in the training dataset. Here we leverage recent advances in
  text-guided image generation, which lead to major breakthroughs in text-guided
  shape generation. We describe an approach called SDS-Complete that uses a
  pre-trained text-to-image diffusion model and leverages the text semantics of a
  given incomplete point cloud of an object, to obtain a complete surface
  representation. SDS-Complete can complete a variety of objects using test-time
  optimization without expensive collection of 3D information. We evaluate SDS
  Complete on incomplete scanned objects, captured by real-world depth sensors
  and LiDAR scanners. We find that it effectively reconstructs objects that are
  absent from common datasets, reducing Chamfer loss by 50% on average compared
  with current methods. Project page: https://sds-complete.github.io/
paper_ai_keywords: text-guided image generation text-guided shape generation text-to-image diffusion model text semantics