Concept Lancet: Image Editing with Compositional Representation
  Transplant
Diffusion models are widely used for image editing tasks. Existing editing
methods often design a representation manipulation procedure by curating an
edit direction in the text embedding or score space. However, such a procedure
faces a key challenge: overestimating the edit strength harms visual
consistency while underestimating it fails the editing task. Notably, each
source image may require a different editing strength, and it is costly to
search for an appropriate strength via trial-and-error. To address this
challenge, we propose Concept Lancet (CoLan), a zero-shot plug-and-play
framework for principled representation manipulation in diffusion-based image
editing. At inference time, we decompose the source input in the latent (text
embedding or diffusion score) space as a sparse linear combination of the
representations of the collected visual concepts. This allows us to accurately
estimate the presence of concepts in each image, which informs the edit. Based
on the editing task (replace/add/remove), we perform a customized concept
transplant process to impose the corresponding editing direction. To
sufficiently model the concept space, we curate a conceptual representation
dataset, CoLan-150K, which contains diverse descriptions and scenarios of
visual terms and phrases for the latent dictionary. Experiments on multiple
diffusion-based image editing baselines show that methods equipped with CoLan
achieve state-of-the-art performance in editing effectiveness and consistency
preservation.
67f01efe81f4f7a1b43f4930
Jinqi Luo
claimed_verified
2025-04-06T08:11:28.131Z
6279a4f6812ee439d9c72d3f
/avatars/a35a5674d1168d345d9fc5018485283e.svg
Jinqi Luo
user
peterljq
67f01efe81f4f7a1b43f4931
Tianjiao Ding
67f01efe81f4f7a1b43f4932
Kwan Ho Ryan Chan
admin_assigned
2025-04-08T09:13:58.769Z
6627043551cedbbb0b352047
/avatars/8c4f80a24f8ba8761d33692c4ed28c29.svg
Kwan Ho Ryan Chan
user
ryanckh
67f01efe81f4f7a1b43f4933
Hancheng Min
67f01efe81f4f7a1b43f4934
Chris Callison-Burch
admin_assigned
2025-04-08T10:33:20.017Z
6303ce25fc783bfc744216af
/avatars/09f5e87c1f56a1b7f6ef9c5037682285.svg
Chris Callison-Burch
user
CCB
67f01efe81f4f7a1b43f4935
René Vidal
Concept Lancet: Image Editing with Compositional Representation
  Transplant
Diffusion models are widely used for image editing tasks. Existing editing
methods often design a representation manipulation procedure by curating an
edit direction in the text embedding or score space. However, such a procedure
faces a key challenge: overestimating the edit strength harms visual
consistency while underestimating it fails the editing task. Notably, each
source image may require a different editing strength, and it is costly to
search for an appropriate strength via trial-and-error. To address this
challenge, we propose Concept Lancet (CoLan), a zero-shot plug-and-play
framework for principled representation manipulation in diffusion-based image
editing. At inference time, we decompose the source input in the latent (text
embedding or diffusion score) space as a sparse linear combination of the
representations of the collected visual concepts. This allows us to accurately
estimate the presence of concepts in each image, which informs the edit. Based
on the editing task (replace/add/remove), we perform a customized concept
transplant process to impose the corresponding editing direction. To
sufficiently model the concept space, we curate a conceptual representation
dataset, CoLan-150K, which contains diverse descriptions and scenarios of
visual terms and phrases for the latent dictionary. Experiments on multiple
diffusion-based image editing baselines show that methods equipped with CoLan
achieve state-of-the-art performance in editing effectiveness and consistency
preservation.
Concept Lancet (CoLan) is a framework for diffusion-based image editing that improves performance and consistency by decomposing source images into latent concept representations.
diffusion models representation manipulation edit direction latent space text embedding diffusion score sparse linear combination visual concepts concept transplant CoLan-150K conceptual representation dataset