LEDITS: Real Image Editing with DDPM Inversion and Semantic Guidance
Recent large-scale text-guided diffusion models provide powerful
image-generation capabilities. Currently, a significant effort is given to
enable the modification of these images using text only as means to offer
intuitive and versatile editing. However, editing proves to be difficult for
these generative models due to the inherent nature of editing techniques, which
involves preserving certain content from the original image. Conversely, in
text-based models, even minor modifications to the text prompt frequently
result in an entirely distinct result, making attaining one-shot generation
that accurately corresponds to the users intent exceedingly challenging. In
addition, to edit a real image using these state-of-the-art tools, one must
first invert the image into the pre-trained models domain - adding another
factor affecting the edit quality, as well as latency. In this exploratory
report, we propose LEDITS - a combined lightweight approach for real-image
editing, incorporating the Edit Friendly DDPM inversion technique with Semantic
Guidance, thus extending Semantic Guidance to real image editing, while
harnessing the editing capabilities of DDPM inversion as well. This approach
achieves versatile edits, both subtle and extensive as well as alterations in
composition and style, while requiring no optimization nor extensions to the
architecture.
64a392bafbd6fdd06d7dfcac
Linoy Tsaban
admin_assigned
2023-07-04T07:54:34.268Z
638f308fc4444c6ca870b60a
https://cdn-avatars.huggingface.co/v1/production/uploads/638f308fc4444c6ca870b60a/Q11NK-8-JbiilJ-vk2LAR.png
Linoy Tsaban
user
linoyts
64a392bafbd6fdd06d7dfcad
Apolinário Passos
admin_assigned
2023-07-04T07:54:20.772Z
624bebf604abc7ebb01789af
https://cdn-avatars.huggingface.co/v1/production/uploads/1649143001781-624bebf604abc7ebb01789af.jpeg
Apolinário from multimodal AI art
user
multimodalart
LEDITS: Real Image Editing with DDPM Inversion and Semantic Guidance
Recent large-scale text-guided diffusion models provide powerful
image-generation capabilities. Currently, a significant effort is given to
enable the modification of these images using text only as means to offer
intuitive and versatile editing. However, editing proves to be difficult for
these generative models due to the inherent nature of editing techniques, which
involves preserving certain content from the original image. Conversely, in
text-based models, even minor modifications to the text prompt frequently
result in an entirely distinct result, making attaining one-shot generation
that accurately corresponds to the users intent exceedingly challenging. In
addition, to edit a real image using these state-of-the-art tools, one must
first invert the image into the pre-trained models domain - adding another
factor affecting the edit quality, as well as latency. In this exploratory
report, we propose LEDITS - a combined lightweight approach for real-image
editing, incorporating the Edit Friendly DDPM inversion technique with Semantic
Guidance, thus extending Semantic Guidance to real image editing, while
harnessing the editing capabilities of DDPM inversion as well. This approach
achieves versatile edits, both subtle and extensive as well as alterations in
composition and style, while requiring no optimization nor extensions to the
architecture.
LEDITS combines Edit Friendly DDPM inversion and Semantic Guidance to enable versatile and efficient real-image editing using text prompts.
diffusion models image-generation text-guided editing DDPM inversion Semantic Guidance real-image editing composition style