TurboEdit: Instant text-based image editing
We address the challenges of precise image inversion and disentangled image
editing in the context of few-step diffusion models. We introduce an encoder
based iterative inversion technique. The inversion network is conditioned on
the input image and the reconstructed image from the previous step, allowing
for correction of the next reconstruction towards the input image. We
demonstrate that disentangled controls can be easily achieved in the few-step
diffusion model by conditioning on an (automatically generated) detailed text
prompt. To manipulate the inverted image, we freeze the noise maps and modify
one attribute in the text prompt (either manually or via instruction based
editing driven by an LLM), resulting in the generation of a new image similar
to the input image with only one attribute changed. It can further control the
editing strength and accept instructive text prompt. Our approach facilitates
realistic text-guided image edits in real-time, requiring only 8 number of
functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit.
Our method is not only fast, but also significantly outperforms
state-of-the-art multi-step diffusion editing techniques.
66c2a599a545b0cefe694abd
Zongze Wu
66c2a599a545b0cefe694abe
Nicholas Kolkin
66c2a599a545b0cefe694abf
Jonathan Brandt
66c2a599a545b0cefe694ac0
Richard Zhang
66c2a599a545b0cefe694ac1
Eli Shechtman
TurboEdit: Instant text-based image editing
We address the challenges of precise image inversion and disentangled image
editing in the context of few-step diffusion models. We introduce an encoder
based iterative inversion technique. The inversion network is conditioned on
the input image and the reconstructed image from the previous step, allowing
for correction of the next reconstruction towards the input image. We
demonstrate that disentangled controls can be easily achieved in the few-step
diffusion model by conditioning on an (automatically generated) detailed text
prompt. To manipulate the inverted image, we freeze the noise maps and modify
one attribute in the text prompt (either manually or via instruction based
editing driven by an LLM), resulting in the generation of a new image similar
to the input image with only one attribute changed. It can further control the
editing strength and accept instructive text prompt. Our approach facilitates
realistic text-guided image edits in real-time, requiring only 8 number of
functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit.
Our method is not only fast, but also significantly outperforms
state-of-the-art multi-step diffusion editing techniques.
The paper presents an iterative inversion technique using few-step diffusion models, enabling precise image inversion and disentangled editing through text prompts, achieving real-time edits with low functional evaluations and superior performance over existing methods.
few-step diffusion models iterative inversion disentangled image editing encoder-based technique reconstruction detailed text prompt noise maps instruction-based editing LLM functional evaluations text-guided image edits