DATENeRF: Depth-Aware Text-based Editing of NeRFs
Recent advancements in diffusion models have shown remarkable proficiency in
editing 2D images based on text prompts. However, extending these techniques to
edit scenes in Neural Radiance Fields (NeRF) is complex, as editing individual
2D frames can result in inconsistencies across multiple views. Our crucial
insight is that a NeRF scene's geometry can serve as a bridge to integrate
these 2D edits. Utilizing this geometry, we employ a depth-conditioned
ControlNet to enhance the coherence of each 2D image modification. Moreover, we
introduce an inpainting approach that leverages the depth information of NeRF
scenes to distribute 2D edits across different images, ensuring robustness
against errors and resampling challenges. Our results reveal that this
methodology achieves more consistent, lifelike, and detailed edits than
existing leading methods for text-driven NeRF scene editing.
6614c2a106ee61ff24cc8914
Sara Rojas
6614c2a106ee61ff24cc8915
Julien Philip
claimed_verified
2024-06-26T08:36:08.164Z
63e0dbb35ba41def8794dab8
/avatars/38a28103b15d93c7d9243bc5418e623e.svg
Julien Philip
user
Jovphi
6614c2a106ee61ff24cc8916
Kai Zhang
6614c2a106ee61ff24cc8917
Sai Bi
6614c2a106ee61ff24cc8918
Fujun Luan
6614c2a106ee61ff24cc8919
Bernard Ghanem
6614c2a106ee61ff24cc891a
Kalyan Sunkavall
DATENeRF: Depth-Aware Text-based Editing of NeRFs
Recent advancements in diffusion models have shown remarkable proficiency in
editing 2D images based on text prompts. However, extending these techniques to
edit scenes in Neural Radiance Fields (NeRF) is complex, as editing individual
2D frames can result in inconsistencies across multiple views. Our crucial
insight is that a NeRF scene's geometry can serve as a bridge to integrate
these 2D edits. Utilizing this geometry, we employ a depth-conditioned
ControlNet to enhance the coherence of each 2D image modification. Moreover, we
introduce an inpainting approach that leverages the depth information of NeRF
scenes to distribute 2D edits across different images, ensuring robustness
against errors and resampling challenges. Our results reveal that this
methodology achieves more consistent, lifelike, and detailed edits than
existing leading methods for text-driven NeRF scene editing.
Depth-conditioned ControlNet and inpainting approach ensure consistent and detailed text-driven edits in NeRF scenes by leveraging geometric information.
diffusion models Neural Radiance Fields (NeRF) depth-conditioned ControlNet inpainting 2D image modification geometric information