Pathways on the Image Manifold: Image Editing via Video Generation
Recent advances in image editing, driven by image diffusion models, have
shown remarkable progress. However, significant challenges remain, as these
models often struggle to follow complex edit instructions accurately and
frequently compromise fidelity by altering key elements of the original image.
Simultaneously, video generation has made remarkable strides, with models that
effectively function as consistent and continuous world simulators. In this
paper, we propose merging these two fields by utilizing image-to-video models
for image editing. We reformulate image editing as a temporal process, using
pretrained video models to create smooth transitions from the original image to
the desired edit. This approach traverses the image manifold continuously,
ensuring consistent edits while preserving the original image's key aspects.
Our approach achieves state-of-the-art results on text-based image editing,
demonstrating significant improvements in both edit accuracy and image
preservation.
6746c37efb7a1de1dca06a6d
Noam Rotstein
claimed_verified
2024-11-27T22:19:56.306Z
62b3e85bcbd2a402fc7804b1
/avatars/63125ce8a1e20b8c6e836f223d24284f.svg
noam rotstein
user
noamrot
6746c37efb7a1de1dca06a6e
Gal Yona
6746c37efb7a1de1dca06a6f
Daniel Silver
6746c37efb7a1de1dca06a70
Roy Velich
claimed_verified
2024-11-29T10:08:14.111Z
63ecaf7460ff4b318ad03ebb
/avatars/7a5bf1854f1eae9cc5fd8392a3f9fba3.svg
Roy Velich
user
royve
6746c37efb7a1de1dca06a71
David Bensaïd
6746c37efb7a1de1dca06a72
Ron Kimmel
Pathways on the Image Manifold: Image Editing via Video Generation
Recent advances in image editing, driven by image diffusion models, have
shown remarkable progress. However, significant challenges remain, as these
models often struggle to follow complex edit instructions accurately and
frequently compromise fidelity by altering key elements of the original image.
Simultaneously, video generation has made remarkable strides, with models that
effectively function as consistent and continuous world simulators. In this
paper, we propose merging these two fields by utilizing image-to-video models
for image editing. We reformulate image editing as a temporal process, using
pretrained video models to create smooth transitions from the original image to
the desired edit. This approach traverses the image manifold continuously,
ensuring consistent edits while preserving the original image's key aspects.
Our approach achieves state-of-the-art results on text-based image editing,
demonstrating significant improvements in both edit accuracy and image
preservation.
Image editing is improved using pretrained video models to ensure accurate edits and preserve image fidelity by reformulating the process as a smooth temporal transition.
image diffusion models image-to-video models temporal process pretrained video models image manifold text-based image editing