UIP2P: Unsupervised Instruction-based Image Editing via Cycle Edit
  Consistency
We propose an unsupervised model for instruction-based image editing that
eliminates the need for ground-truth edited images during training. Existing
supervised methods depend on datasets containing triplets of input image,
edited image, and edit instruction. These are generated by either existing
editing methods or human-annotations, which introduce biases and limit their
generalization ability. Our method addresses these challenges by introducing a
novel editing mechanism called Cycle Edit Consistency (CEC), which applies
forward and backward edits in one training step and enforces consistency in
image and attention spaces. This allows us to bypass the need for ground-truth
edited images and unlock training for the first time on datasets comprising
either real image-caption pairs or image-caption-edit triplets. We empirically
show that our unsupervised technique performs better across a broader range of
edits with high fidelity and precision. By eliminating the need for
pre-existing datasets of triplets, reducing biases associated with supervised
methods, and proposing CEC, our work represents a significant advancement in
unblocking scaling of instruction-based image editing.
6764deae8ae9bee011733928
Enis Simsar
claimed_verified
2024-12-20T08:32:47.269Z
63412f2add8853dc7e306a4f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/nqbteeQhS2dPA5LVy7dTD.png
Enis Simsar
user
enisimsar
6764deae8ae9bee011733929
Alessio Tonioni
admin_assigned
2024-12-20T09:28:56.892Z
63402b30e670ff9cf63d8caa
/avatars/0aee84d132a78d4ec71663836a57a245.svg
Alessio Tonioni
user
Alessiot
6764deae8ae9bee01173392a
Yongqin Xian
6764deae8ae9bee01173392b
Thomas Hofmann
admin_assigned
2024-12-20T09:28:42.685Z
6630831ce888d89069e6276a
/avatars/b40d00ac8978405dd2ae66166ac969ba.svg
Thomas Hofmann
user
thofmann
6764deae8ae9bee01173392c
Federico Tombari
UIP2P: Unsupervised Instruction-based Image Editing via Cycle Edit
  Consistency
We propose an unsupervised model for instruction-based image editing that
eliminates the need for ground-truth edited images during training. Existing
supervised methods depend on datasets containing triplets of input image,
edited image, and edit instruction. These are generated by either existing
editing methods or human-annotations, which introduce biases and limit their
generalization ability. Our method addresses these challenges by introducing a
novel editing mechanism called Cycle Edit Consistency (CEC), which applies
forward and backward edits in one training step and enforces consistency in
image and attention spaces. This allows us to bypass the need for ground-truth
edited images and unlock training for the first time on datasets comprising
either real image-caption pairs or image-caption-edit triplets. We empirically
show that our unsupervised technique performs better across a broader range of
edits with high fidelity and precision. By eliminating the need for
pre-existing datasets of triplets, reducing biases associated with supervised
methods, and proposing CEC, our work represents a significant advancement in
unblocking scaling of instruction-based image editing.
Cycle Edit Consistency allows unsupervised instruction-based image editing by enforcing consistency in image and attention spaces, enabling training on real image-caption pairs without ground-truth edited images.
Cycle Edit Consistency unsupervised model forward and backward edits image space attention space real image-caption pairs edit instruction instruction-based image editing