FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videos
<|reference_start|>FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videos: Digitising the 3D world into a clean, CAD model-based representation has important applications for augmented reality and robotics. Current state-of-the-art methods are computationally intensive as they individually encode each detected object and optimise CAD alignments in a second stage. In this work, we propose FastCAD, a real-time method that simultaneously retrieves and aligns CAD models for all objects in a given scene. In contrast to previous works, we directly predict alignment parameters and shape embeddings. We achieve high-quality shape retrievals by learning CAD embeddings in a contrastive learning framework and distilling those into FastCAD. Our single-stage method accelerates the inference time by a factor of 50 compared to other methods operating on RGB-D scans while outperforming them on the challenging Scan2CAD alignment benchmark. Further, our approach collaborates seamlessly with online 3D reconstruction techniques. This enables the real-time generation of precise CAD model-based reconstructions from videos at 10 FPS. Doing so, we significantly improve the Scan2CAD alignment accuracy in the video setting from 43.0% to 48.2% and the reconstruction accuracy from 22.9% to 29.6%.<|reference_end|>
@article{langer2024fastcad:,
    title={FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videos},
    author={Florian Langer, Jihong Ju, Georgi Dikov, Gerhard Reitmayr, Mohsen
  Ghafoorian},
    journal={arXiv preprint arXiv:2403.15161},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.15161},
    primaryClass={cs.CV}
}