DreamMix: Decoupling Object Attributes for Enhanced Editability in
  Customized Image Inpainting
Subject-driven image inpainting has emerged as a popular task in image
editing alongside recent advancements in diffusion models. Previous methods
primarily focus on identity preservation but struggle to maintain the
editability of inserted objects. In response, this paper introduces DreamMix, a
diffusion-based generative model adept at inserting target objects into given
scenes at user-specified locations while concurrently enabling arbitrary
text-driven modifications to their attributes. In particular, we leverage
advanced foundational inpainting models and introduce a disentangled
local-global inpainting framework to balance precise local object insertion
with effective global visual coherence. Additionally, we propose an Attribute
Decoupling Mechanism (ADM) and a Textual Attribute Substitution (TAS) module to
improve the diversity and discriminative capability of the text-based attribute
guidance, respectively. Extensive experiments demonstrate that DreamMix
effectively balances identity preservation and attribute editability across
various application scenarios, including object insertion, attribute editing,
and small object inpainting. Our code is publicly available at
https://github.com/mycfhs/DreamMix.
6746801be0ff542317fa9902
Yicheng Yang
6746801be0ff542317fa9903
Pengxiang Li
claimed_verified
2025-01-09T10:07:39.672Z
64245f2c089d5fae56b4549a
https://cdn-avatars.huggingface.co/v1/production/uploads/64245f2c089d5fae56b4549a/qUHFsL9Svwyj5BKpfMtaY.jpeg
Pengxiang Li
user
pengxiang
6746801be0ff542317fa9904
Lu Zhang
6746801be0ff542317fa9905
Liqian Ma
6746801be0ff542317fa9906
Ping Hu
6746801be0ff542317fa9907
Siyu Du
6746801be0ff542317fa9908
Yunzhi Zhuge
6746801be0ff542317fa9909
Xu Jia
6746801be0ff542317fa990a
Huchuan Lu
DreamMix: Decoupling Object Attributes for Enhanced Editability in
  Customized Image Inpainting
Subject-driven image inpainting has emerged as a popular task in image
editing alongside recent advancements in diffusion models. Previous methods
primarily focus on identity preservation but struggle to maintain the
editability of inserted objects. In response, this paper introduces DreamMix, a
diffusion-based generative model adept at inserting target objects into given
scenes at user-specified locations while concurrently enabling arbitrary
text-driven modifications to their attributes. In particular, we leverage
advanced foundational inpainting models and introduce a disentangled
local-global inpainting framework to balance precise local object insertion
with effective global visual coherence. Additionally, we propose an Attribute
Decoupling Mechanism (ADM) and a Textual Attribute Substitution (TAS) module to
improve the diversity and discriminative capability of the text-based attribute
guidance, respectively. Extensive experiments demonstrate that DreamMix
effectively balances identity preservation and attribute editability across
various application scenarios, including object insertion, attribute editing,
and small object inpainting. Our code is publicly available at
https://github.com/mycfhs/DreamMix.
DreamMix is a diffusion-based generative model that inserts objects into images and allows for attribute modification using text, maintaining identity and visual coherence.
diffusion-based generative model disentangled local-global inpainting framework Attribute Decoupling Mechanism (ADM) Textual Attribute Substitution (TAS module) object insertion attribute editing small object inpainting