NeRF-Insert: 3D Local Editing with Multimodal Control Signals
<|reference_start|>NeRF-Insert: 3D Local Editing with Multimodal Control Signals: We propose NeRF-Insert, a NeRF editing framework that allows users to make high-quality local edits with a flexible level of control. Unlike previous work that relied on image-to-image models, we cast scene editing as an in-painting problem, which encourages the global structure of the scene to be preserved. Moreover, while most existing methods use only textual prompts to condition edits, our framework accepts a combination of inputs of different modalities as reference. More precisely, a user may provide a combination of textual and visual inputs including images, CAD models, and binary image masks for specifying a 3D region. We use generic image generation models to in-paint the scene from multiple viewpoints, and lift the local edits to a 3D-consistent NeRF edit. Compared to previous methods, our results show better visual quality and also maintain stronger consistency with the original NeRF.<|reference_end|>
@article{sabat2024nerf-insert:,
    title={NeRF-Insert: 3D Local Editing with Multimodal Control Signals},
    author={Benet Oriol Sabat, Alessandro Achille, Matthew Trager, Stefano Soatto},
    journal={arXiv preprint arXiv:2404.19204},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.19204},
    primaryClass={cs.CV cs.AI cs.GR}
}