Editable Image Elements for Controllable Synthesis
Diffusion models have made significant advances in text-guided synthesis
tasks. However, editing user-provided images remains challenging, as the high
dimensional noise input space of diffusion models is not naturally suited for
image inversion or spatial editing. In this work, we propose an image
representation that promotes spatial editing of input images using a diffusion
model. Concretely, we learn to encode an input into "image elements" that can
faithfully reconstruct an input image. These elements can be intuitively edited
by a user, and are decoded by a diffusion model into realistic images. We show
the effectiveness of our representation on various image editing tasks, such as
object resizing, rearrangement, dragging, de-occlusion, removal, variation, and
image composition. Project page:
https://jitengmu.github.io/Editable_Image_Elements/
662a967ce58d277c81d53e53
Jiteng Mu
admin_assigned
2024-04-26T12:38:28.988Z
63a38a4a2262903957e12f14
/avatars/ad0e98736bdba5fa7a64c15c876c67a4.svg
JitengMu
user
JitengMu
662a967ce58d277c81d53e54
Michaël Gharbi
admin_assigned
2024-04-26T12:38:39.879Z
658161ad208369a9403ca9a1
/avatars/e2324ab2a712100681b0b8127305212b.svg
Michael Gharbi
user
mgharbi
662a967ce58d277c81d53e55
Richard Zhang
662a967ce58d277c81d53e56
Eli Shechtman
662a967ce58d277c81d53e57
Nuno Vasconcelos
662a967ce58d277c81d53e58
Xiaolong Wang
662a967ce58d277c81d53e59
Taesung Park
admin_assigned
2024-04-26T12:50:00.655Z
637563c59ef4e3b87a60d503
/avatars/8264de115e6305e4720c78fc7dce7f8d.svg
Taesung Park
user
taesungp
Editable Image Elements for Controllable Synthesis
Diffusion models have made significant advances in text-guided synthesis
tasks. However, editing user-provided images remains challenging, as the high
dimensional noise input space of diffusion models is not naturally suited for
image inversion or spatial editing. In this work, we propose an image
representation that promotes spatial editing of input images using a diffusion
model. Concretely, we learn to encode an input into "image elements" that can
faithfully reconstruct an input image. These elements can be intuitively edited
by a user, and are decoded by a diffusion model into realistic images. We show
the effectiveness of our representation on various image editing tasks, such as
object resizing, rearrangement, dragging, de-occlusion, removal, variation, and
image composition. Project page:
https://jitengmu.github.io/Editable_Image_Elements/
A new image representation enables spatial editing of images using diffusion models by encoding them into editable elements that produce realistic outputs when decoded.
diffusion models image representation image elements image inversion spatial editing object resizing rearrangement dragging de-occlusion removal variation image composition