title: DATENeRF: Depth-Aware Text-based Editing of NeRFs
summary: Recent advancements in diffusion models have shown remarkable proficiency in
  editing 2D images based on text prompts. However, extending these techniques to
  edit scenes in Neural Radiance Fields (NeRF) is complex, as editing individual
  2D frames can result in inconsistencies across multiple views. Our crucial
  insight is that a NeRF scene's geometry can serve as a bridge to integrate
  these 2D edits. Utilizing this geometry, we employ a depth-conditioned
  ControlNet to enhance the coherence of each 2D image modification. Moreover, we
  introduce an inpainting approach that leverages the depth information of NeRF
  scenes to distribute 2D edits across different images, ensuring robustness
  against errors and resampling challenges. Our results reveal that this
  methodology achieves more consistent, lifelike, and detailed edits than
  existing leading methods for text-driven NeRF scene editing.
paper_authors._id: 6614c2a106ee61ff24cc8914
paper_authors.name: Sara Rojas
paper_authors._id: 6614c2a106ee61ff24cc8915
paper_authors.name: Julien Philip
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-26T08:36:08.164Z
paper_authors._id: 63e0dbb35ba41def8794dab8
paper_authors.avatarUrl: /avatars/38a28103b15d93c7d9243bc5418e623e.svg
paper_authors.fullname: Julien Philip
paper_authors.type: user
paper_authors.user: Jovphi
paper_authors._id: 6614c2a106ee61ff24cc8916
paper_authors.name: Kai Zhang
paper_authors._id: 6614c2a106ee61ff24cc8917
paper_authors.name: Sai Bi
paper_authors._id: 6614c2a106ee61ff24cc8918
paper_authors.name: Fujun Luan
paper_authors._id: 6614c2a106ee61ff24cc8919
paper_authors.name: Bernard Ghanem
paper_authors._id: 6614c2a106ee61ff24cc891a
paper_authors.name: Kalyan Sunkavall
paper_title: DATENeRF: Depth-Aware Text-based Editing of NeRFs
paper_summary: Recent advancements in diffusion models have shown remarkable proficiency in
  editing 2D images based on text prompts. However, extending these techniques to
  edit scenes in Neural Radiance Fields (NeRF) is complex, as editing individual
  2D frames can result in inconsistencies across multiple views. Our crucial
  insight is that a NeRF scene's geometry can serve as a bridge to integrate
  these 2D edits. Utilizing this geometry, we employ a depth-conditioned
  ControlNet to enhance the coherence of each 2D image modification. Moreover, we
  introduce an inpainting approach that leverages the depth information of NeRF
  scenes to distribute 2D edits across different images, ensuring robustness
  against errors and resampling challenges. Our results reveal that this
  methodology achieves more consistent, lifelike, and detailed edits than
  existing leading methods for text-driven NeRF scene editing.
paper_ai_summary: Depth-conditioned ControlNet and inpainting approach ensure consistent and detailed text-driven edits in NeRF scenes by leveraging geometric information.
paper_ai_keywords: diffusion models Neural Radiance Fields (NeRF) depth-conditioned ControlNet inpainting 2D image modification geometric information