title: Tuning-Free Image Editing with Fidelity and Editability via Unified
    Latent Diffusion Model
summary: Balancing fidelity and editability is essential in text-based image editing
  (TIE), where failures commonly lead to over- or under-editing issues. Existing
  methods typically rely on attention injections for structure preservation and
  leverage the inherent text alignment capabilities of pre-trained text-to-image
  (T2I) models for editability, but they lack explicit and unified mechanisms to
  properly balance these two objectives. In this work, we introduce UnifyEdit, a
  tuning-free method that performs diffusion latent optimization to enable a
  balanced integration of fidelity and editability within a unified framework.
  Unlike direct attention injections, we develop two attention-based constraints:
  a self-attention (SA) preservation constraint for structural fidelity, and a
  cross-attention (CA) alignment constraint to enhance text alignment for
  improved editability. However, simultaneously applying both constraints can
  lead to gradient conflicts, where the dominance of one constraint results in
  over- or under-editing. To address this challenge, we introduce an adaptive
  time-step scheduler that dynamically adjusts the influence of these
  constraints, guiding the diffusion latent toward an optimal balance. Extensive
  quantitative and qualitative experiments validate the effectiveness of our
  approach, demonstrating its superiority in achieving a robust balance between
  structure preservation and text alignment across various editing tasks,
  outperforming other state-of-the-art methods. The source code will be available
  at https://github.com/CUC-MIPG/UnifyEdit.
paper_authors._id: 67f5dc86015730c161ce291b
paper_authors.name: Qi Mao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-10T06:45:41.166Z
paper_authors._id: 6388a7e98a5dbe2f3dc61faa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6388a7e98a5dbe2f3dc61faa/Zj8pR7BG_wOQjsYzosb5p.jpeg
paper_authors.fullname: Qi Mao
paper_authors.type: user
paper_authors.user: HelenMao
paper_authors._id: 67f5dc86015730c161ce291c
paper_authors.name: Lan Chen
paper_authors._id: 67f5dc86015730c161ce291d
paper_authors.name: Yuchao Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:43:17.644Z
paper_authors._id: 63021630a35b21bd8a53305a
paper_authors.avatarUrl: /avatars/7a7e8b39749eda61e57d8a1908726558.svg
paper_authors.fullname: Gu Yuchao
paper_authors.type: user
paper_authors.user: guyuchao
paper_authors._id: 67f5dc86015730c161ce291e
paper_authors.name: Mike Zheng Shou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:43:35.750Z
paper_authors._id: 661ab3da2b14565c7acccf5c
paper_authors.avatarUrl: /avatars/fa4fc03664803e02aede4d4c3d50b393.svg
paper_authors.fullname: Mike Zheng Shou
paper_authors.type: user
paper_authors.user: AnalMom
paper_authors._id: 67f5dc86015730c161ce291f
paper_authors.name: Ming-Hsuan Yang
paper_title: Tuning-Free Image Editing with Fidelity and Editability via Unified
    Latent Diffusion Model
paper_summary: Balancing fidelity and editability is essential in text-based image editing
  (TIE), where failures commonly lead to over- or under-editing issues. Existing
  methods typically rely on attention injections for structure preservation and
  leverage the inherent text alignment capabilities of pre-trained text-to-image
  (T2I) models for editability, but they lack explicit and unified mechanisms to
  properly balance these two objectives. In this work, we introduce UnifyEdit, a
  tuning-free method that performs diffusion latent optimization to enable a
  balanced integration of fidelity and editability within a unified framework.
  Unlike direct attention injections, we develop two attention-based constraints:
  a self-attention (SA) preservation constraint for structural fidelity, and a
  cross-attention (CA) alignment constraint to enhance text alignment for
  improved editability. However, simultaneously applying both constraints can
  lead to gradient conflicts, where the dominance of one constraint results in
  over- or under-editing. To address this challenge, we introduce an adaptive
  time-step scheduler that dynamically adjusts the influence of these
  constraints, guiding the diffusion latent toward an optimal balance. Extensive
  quantitative and qualitative experiments validate the effectiveness of our
  approach, demonstrating its superiority in achieving a robust balance between
  structure preservation and text alignment across various editing tasks,
  outperforming other state-of-the-art methods. The source code will be available
  at https://github.com/CUC-MIPG/UnifyEdit.
paper_ai_summary: UnifyEdit uses diffusion latent optimization with adaptive constraints to balance structural fidelity and text alignment in text-based image editing.
paper_ai_keywords: diffusion latent optimization self-attention cross-attention adaptive time-step scheduler structure preservation text alignment text-based image editing T2I models over-editing under-editing