DreamEditor: Text-Driven 3D Scene Editing with Neural Fields
Neural fields have achieved impressive advancements in view synthesis and
scene reconstruction. However, editing these neural fields remains challenging
due to the implicit encoding of geometry and texture information. In this
paper, we propose DreamEditor, a novel framework that enables users to perform
controlled editing of neural fields using text prompts. By representing scenes
as mesh-based neural fields, DreamEditor allows localized editing within
specific regions. DreamEditor utilizes the text encoder of a pretrained
text-to-Image diffusion model to automatically identify the regions to be
edited based on the semantics of the text prompts. Subsequently, DreamEditor
optimizes the editing region and aligns its geometry and texture with the text
prompts through score distillation sampling [29]. Extensive experiments have
demonstrated that DreamEditor can accurately edit neural fields of real-world
scenes according to the given text prompts while ensuring consistency in
irrelevant areas. DreamEditor generates highly realistic textures and geometry,
significantly surpassing previous works in both quantitative and qualitative
evaluations.
64991c2b49fa1e7cccc97aad
Jingyu Zhuang
admin_assigned
2023-06-26T08:57:06.210Z
63611112108cc69e687b8ea6
/avatars/8c96fa784ce747d3c1c4bcb09bccf9c0.svg
zhuang jingyu
user
zjy526223908
64991c2b49fa1e7cccc97aae
Chen Wang
admin_assigned
2023-06-26T08:57:40.815Z
62e9d5af6687b60b9c01240a
/avatars/3983be5662aa28aefcb18deaf08d7cb1.svg
Chen Wang
user
chenwang
64991c2b49fa1e7cccc97aaf
Lingjie Liu
64991c2b49fa1e7cccc97ab0
Liang Lin
64991c2b49fa1e7cccc97ab1
Guanbin Li
DreamEditor: Text-Driven 3D Scene Editing with Neural Fields
Neural fields have achieved impressive advancements in view synthesis and
scene reconstruction. However, editing these neural fields remains challenging
due to the implicit encoding of geometry and texture information. In this
paper, we propose DreamEditor, a novel framework that enables users to perform
controlled editing of neural fields using text prompts. By representing scenes
as mesh-based neural fields, DreamEditor allows localized editing within
specific regions. DreamEditor utilizes the text encoder of a pretrained
text-to-Image diffusion model to automatically identify the regions to be
edited based on the semantics of the text prompts. Subsequently, DreamEditor
optimizes the editing region and aligns its geometry and texture with the text
prompts through score distillation sampling [29]. Extensive experiments have
demonstrated that DreamEditor can accurately edit neural fields of real-world
scenes according to the given text prompts while ensuring consistency in
irrelevant areas. DreamEditor generates highly realistic textures and geometry,
significantly surpassing previous works in both quantitative and qualitative
evaluations.
DreamEditor enables controlled editing of neural fields using text prompts through mesh-based representations and score distillation sampling, resulting in realistic and consistent scene modifications.
neural fields view synthesis scene reconstruction mesh-based neural fields text-to-Image diffusion model text encoder score distillation sampling