title: UIP2P: Unsupervised Instruction-based Image Editing via Cycle Edit
    Consistency
summary: We propose an unsupervised model for instruction-based image editing that
  eliminates the need for ground-truth edited images during training. Existing
  supervised methods depend on datasets containing triplets of input image,
  edited image, and edit instruction. These are generated by either existing
  editing methods or human-annotations, which introduce biases and limit their
  generalization ability. Our method addresses these challenges by introducing a
  novel editing mechanism called Cycle Edit Consistency (CEC), which applies
  forward and backward edits in one training step and enforces consistency in
  image and attention spaces. This allows us to bypass the need for ground-truth
  edited images and unlock training for the first time on datasets comprising
  either real image-caption pairs or image-caption-edit triplets. We empirically
  show that our unsupervised technique performs better across a broader range of
  edits with high fidelity and precision. By eliminating the need for
  pre-existing datasets of triplets, reducing biases associated with supervised
  methods, and proposing CEC, our work represents a significant advancement in
  unblocking scaling of instruction-based image editing.
paper_authors._id: 6764deae8ae9bee011733928
paper_authors.name: Enis Simsar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-20T08:32:47.269Z
paper_authors._id: 63412f2add8853dc7e306a4f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/nqbteeQhS2dPA5LVy7dTD.png
paper_authors.fullname: Enis Simsar
paper_authors.type: user
paper_authors.user: enisimsar
paper_authors._id: 6764deae8ae9bee011733929
paper_authors.name: Alessio Tonioni
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:28:56.892Z
paper_authors._id: 63402b30e670ff9cf63d8caa
paper_authors.avatarUrl: /avatars/0aee84d132a78d4ec71663836a57a245.svg
paper_authors.fullname: Alessio Tonioni
paper_authors.type: user
paper_authors.user: Alessiot
paper_authors._id: 6764deae8ae9bee01173392a
paper_authors.name: Yongqin Xian
paper_authors._id: 6764deae8ae9bee01173392b
paper_authors.name: Thomas Hofmann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:28:42.685Z
paper_authors._id: 6630831ce888d89069e6276a
paper_authors.avatarUrl: /avatars/b40d00ac8978405dd2ae66166ac969ba.svg
paper_authors.fullname: Thomas Hofmann
paper_authors.type: user
paper_authors.user: thofmann
paper_authors._id: 6764deae8ae9bee01173392c
paper_authors.name: Federico Tombari
paper_title: UIP2P: Unsupervised Instruction-based Image Editing via Cycle Edit
    Consistency
paper_summary: We propose an unsupervised model for instruction-based image editing that
  eliminates the need for ground-truth edited images during training. Existing
  supervised methods depend on datasets containing triplets of input image,
  edited image, and edit instruction. These are generated by either existing
  editing methods or human-annotations, which introduce biases and limit their
  generalization ability. Our method addresses these challenges by introducing a
  novel editing mechanism called Cycle Edit Consistency (CEC), which applies
  forward and backward edits in one training step and enforces consistency in
  image and attention spaces. This allows us to bypass the need for ground-truth
  edited images and unlock training for the first time on datasets comprising
  either real image-caption pairs or image-caption-edit triplets. We empirically
  show that our unsupervised technique performs better across a broader range of
  edits with high fidelity and precision. By eliminating the need for
  pre-existing datasets of triplets, reducing biases associated with supervised
  methods, and proposing CEC, our work represents a significant advancement in
  unblocking scaling of instruction-based image editing.
paper_ai_summary: Cycle Edit Consistency allows unsupervised instruction-based image editing by enforcing consistency in image and attention spaces, enabling training on real image-caption pairs without ground-truth edited images.
paper_ai_keywords: Cycle Edit Consistency unsupervised model forward and backward edits image space attention space real image-caption pairs edit instruction instruction-based image editing