title: RealCustom: Narrowing Real Text Word for Real-Time Open-Domain
    Text-to-Image Customization
summary: Text-to-image customization, which aims to synthesize text-driven images for
  the given subjects, has recently revolutionized content creation. Existing
  works follow the pseudo-word paradigm, i.e., represent the given subjects as
  pseudo-words and then compose them with the given text. However, the inherent
  entangled influence scope of pseudo-words with the given text results in a
  dual-optimum paradox, i.e., the similarity of the given subjects and the
  controllability of the given text could not be optimal simultaneously. We
  present RealCustom that, for the first time, disentangles similarity from
  controllability by precisely limiting subject influence to relevant parts only,
  achieved by gradually narrowing real text word from its general connotation to
  the specific subject and using its cross-attention to distinguish relevance.
  Specifically, RealCustom introduces a novel "train-inference" decoupled
  framework: (1) during training, RealCustom learns general alignment between
  visual conditions to original textual conditions by a novel adaptive scoring
  module to adaptively modulate influence quantity; (2) during inference, a novel
  adaptive mask guidance strategy is proposed to iteratively update the influence
  scope and influence quantity of the given subjects to gradually narrow the
  generation of the real text word. Comprehensive experiments demonstrate the
  superior real-time customization ability of RealCustom in the open domain,
  achieving both unprecedented similarity of the given subjects and
  controllability of the given text for the first time. The project page is
  https://corleone-huang.github.io/realcustom/.
paper_authors._id: 65e5534eef19db6d7416193c
paper_authors.name: Mengqi Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-04T15:05:03.266Z
paper_authors._id: 630636bcd37ce67e0e4d1d42
paper_authors.avatarUrl: /avatars/c3ba695d339eaaf9e810bfa0d9a7689a.svg
paper_authors.fullname: Mengqi Huang
paper_authors.type: user
paper_authors.user: CoreloneH
paper_authors._id: 65e5534eef19db6d7416193d
paper_authors.name: Zhendong Mao
paper_authors._id: 65e5534eef19db6d7416193e
paper_authors.name: Mingcong Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-04T10:24:35.851Z
paper_authors._id: 619b404bab4c7b7f16a7d57d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/619b404bab4c7b7f16a7d57d/coT_UGRfBOUAeSxjyhdlG.jpeg
paper_authors.fullname: Mingcong Liu
paper_authors.type: user
paper_authors.user: onion-liu
paper_authors._id: 65e5534eef19db6d7416193f
paper_authors.name: Qian He
paper_authors._id: 65e5534eef19db6d74161940
paper_authors.name: Yongdong Zhang
paper_title: RealCustom: Narrowing Real Text Word for Real-Time Open-Domain
    Text-to-Image Customization
paper_summary: Text-to-image customization, which aims to synthesize text-driven images for
  the given subjects, has recently revolutionized content creation. Existing
  works follow the pseudo-word paradigm, i.e., represent the given subjects as
  pseudo-words and then compose them with the given text. However, the inherent
  entangled influence scope of pseudo-words with the given text results in a
  dual-optimum paradox, i.e., the similarity of the given subjects and the
  controllability of the given text could not be optimal simultaneously. We
  present RealCustom that, for the first time, disentangles similarity from
  controllability by precisely limiting subject influence to relevant parts only,
  achieved by gradually narrowing real text word from its general connotation to
  the specific subject and using its cross-attention to distinguish relevance.
  Specifically, RealCustom introduces a novel "train-inference" decoupled
  framework: (1) during training, RealCustom learns general alignment between
  visual conditions to original textual conditions by a novel adaptive scoring
  module to adaptively modulate influence quantity; (2) during inference, a novel
  adaptive mask guidance strategy is proposed to iteratively update the influence
  scope and influence quantity of the given subjects to gradually narrow the
  generation of the real text word. Comprehensive experiments demonstrate the
  superior real-time customization ability of RealCustom in the open domain,
  achieving both unprecedented similarity of the given subjects and
  controllability of the given text for the first time. The project page is
  https://corleone-huang.github.io/realcustom/.
paper_ai_summary: RealCustom disentangles similarity and controllability in text-to-image customization by using a decoupled train-inference framework with adaptive scoring and mask guidance strategies.
paper_ai_keywords: text-to-image customization pseudo-words dual-optimum paradox RealCustom cross-attention adaptive scoring module adaptive mask guidance strategy