NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows
<|reference_start|>NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows: We present a method for automatically modifying a NeRF representation based on a single observation of a non-rigid transformed version of the original scene. Our method defines the transformation as a 3D flow, specifically as a weighted linear blending of rigid transformations of 3D anchor points that are defined on the surface of the scene. In order to identify anchor points, we introduce a novel correspondence algorithm that first matches RGB-based pairs, then leverages multi-view information and 3D reprojection to robustly filter false positives in two steps. We also introduce a new dataset for exploring the problem of modifying a NeRF scene through a single observation. Our dataset ( https://github.com/nerfdeformer/nerfdeformer ) contains 113 synthetic scenes leveraging 47 3D assets. We show that our proposed method outperforms NeRF editing methods as well as diffusion-based methods, and we also explore different methods for filtering correspondences.<|reference_end|>
@article{tang2024nerfdeformer:,
    title={NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows},
    author={Zhenggang Tang, Zhongzheng Ren, Xiaoming Zhao, Bowen Wen, Jonathan
  Tremblay, Stan Birchfield, Alexander Schwing},
    journal={arXiv preprint arXiv:2406.10543},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.10543},
    primaryClass={cs.CV cs.AI}
}