title: Towards Learning to Complete Anything in Lidar
summary: We propose CAL (Complete Anything in Lidar) for Lidar-based shape-completion
  in-the-wild. This is closely related to Lidar-based semantic/panoptic scene
  completion. However, contemporary methods can only complete and recognize
  objects from a closed vocabulary labeled in existing Lidar datasets. Different
  to that, our zero-shot approach leverages the temporal context from multi-modal
  sensor sequences to mine object shapes and semantic features of observed
  objects. These are then distilled into a Lidar-only instance-level completion
  and recognition model. Although we only mine partial shape completions, we find
  that our distilled model learns to infer full object shapes from multiple such
  partial observations across the dataset. We show that our model can be prompted
  on standard benchmarks for Semantic and Panoptic Scene Completion, localize
  objects as (amodal) 3D bounding boxes, and recognize objects beyond fixed class
  vocabularies. Our project page is
  https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar
paper_authors._id: 68010305e501b14512a3f5aa
paper_authors.name: Ayca Takmaz
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-18T09:39:50.743Z
paper_authors._id: 6499631f906b89ee66f1fb81
paper_authors.avatarUrl: /avatars/fe27062dfc20cf57169d596457d0ef92.svg
paper_authors.fullname: Ayca Takmaz
paper_authors.type: user
paper_authors.user: aycatakmaz
paper_authors._id: 68010305e501b14512a3f5ab
paper_authors.name: Cristiano Saltori
paper_authors._id: 68010305e501b14512a3f5ac
paper_authors.name: Neehar Peri
paper_authors._id: 68010305e501b14512a3f5ad
paper_authors.name: Tim Meinhardt
paper_authors._id: 68010305e501b14512a3f5ae
paper_authors.name: Riccardo de Lutio
paper_authors._id: 68010305e501b14512a3f5af
paper_authors.name: Laura Leal-Taixé
paper_authors._id: 68010305e501b14512a3f5b0
paper_authors.name: Aljoša Ošep
paper_title: Towards Learning to Complete Anything in Lidar
paper_summary: We propose CAL (Complete Anything in Lidar) for Lidar-based shape-completion
  in-the-wild. This is closely related to Lidar-based semantic/panoptic scene
  completion. However, contemporary methods can only complete and recognize
  objects from a closed vocabulary labeled in existing Lidar datasets. Different
  to that, our zero-shot approach leverages the temporal context from multi-modal
  sensor sequences to mine object shapes and semantic features of observed
  objects. These are then distilled into a Lidar-only instance-level completion
  and recognition model. Although we only mine partial shape completions, we find
  that our distilled model learns to infer full object shapes from multiple such
  partial observations across the dataset. We show that our model can be prompted
  on standard benchmarks for Semantic and Panoptic Scene Completion, localize
  objects as (amodal) 3D bounding boxes, and recognize objects beyond fixed class
  vocabularies. Our project page is
  https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar
paper_ai_summary: CAL uses multi-modal sensor sequences for zero-shot Lidar-based shape completion and recognition, extending beyond fixed class vocabularies.
paper_ai_keywords: Lidar zero-shot semantic scene completion panoptic scene completion multi-modal instance-level shape completion 3D bounding boxes