title: Geometry-Editable and Appearance-Preserving Object Compositon
summary: General object composition (GOC) aims to seamlessly integrate a target object
  into a background scene with desired geometric properties, while simultaneously
  preserving its fine-grained appearance details. Recent approaches derive
  semantic embeddings and integrate them into advanced diffusion models to enable
  geometry-editable generation. However, these highly compact embeddings encode
  only high-level semantic cues and inevitably discard fine-grained appearance
  details. We introduce a Disentangled Geometry-editable and
  Appearance-preserving Diffusion (DGAD) model that first leverages semantic
  embeddings to implicitly capture the desired geometric transformations and then
  employs a cross-attention retrieval mechanism to align fine-grained appearance
  features with the geometry-edited representation, facilitating both precise
  geometry editing and faithful appearance preservation in object composition.
  Specifically, DGAD builds on CLIP/DINO-derived and reference networks to
  extract semantic embeddings and appearance-preserving representations, which
  are then seamlessly integrated into the encoding and decoding pipelines in a
  disentangled manner. We first integrate the semantic embeddings into
  pre-trained diffusion models that exhibit strong spatial reasoning capabilities
  to implicitly capture object geometry, thereby facilitating flexible object
  manipulation and ensuring effective editability. Then, we design a dense
  cross-attention mechanism that leverages the implicitly learned object geometry
  to retrieve and spatially align appearance features with their corresponding
  regions, ensuring faithful appearance consistency. Extensive experiments on
  public benchmarks demonstrate the effectiveness of the proposed DGAD framework.
paper_authors._id: 68425a585738dda052ea4c91
paper_authors.name: Jianman Lin
paper_authors._id: 68425a585738dda052ea4c92
paper_authors.name: Haojie Li
paper_authors._id: 68425a585738dda052ea4c93
paper_authors.name: Chunmei Qing
paper_authors._id: 68425a585738dda052ea4c94
paper_authors.name: Zhijing Yang
paper_authors._id: 68425a585738dda052ea4c95
paper_authors.name: Liang Lin
paper_authors._id: 68425a585738dda052ea4c96
paper_authors.name: Tianshui Chen
paper_title: Geometry-Editable and Appearance-Preserving Object Compositon
paper_summary: General object composition (GOC) aims to seamlessly integrate a target object
  into a background scene with desired geometric properties, while simultaneously
  preserving its fine-grained appearance details. Recent approaches derive
  semantic embeddings and integrate them into advanced diffusion models to enable
  geometry-editable generation. However, these highly compact embeddings encode
  only high-level semantic cues and inevitably discard fine-grained appearance
  details. We introduce a Disentangled Geometry-editable and
  Appearance-preserving Diffusion (DGAD) model that first leverages semantic
  embeddings to implicitly capture the desired geometric transformations and then
  employs a cross-attention retrieval mechanism to align fine-grained appearance
  features with the geometry-edited representation, facilitating both precise
  geometry editing and faithful appearance preservation in object composition.
  Specifically, DGAD builds on CLIP/DINO-derived and reference networks to
  extract semantic embeddings and appearance-preserving representations, which
  are then seamlessly integrated into the encoding and decoding pipelines in a
  disentangled manner. We first integrate the semantic embeddings into
  pre-trained diffusion models that exhibit strong spatial reasoning capabilities
  to implicitly capture object geometry, thereby facilitating flexible object
  manipulation and ensuring effective editability. Then, we design a dense
  cross-attention mechanism that leverages the implicitly learned object geometry
  to retrieve and spatially align appearance features with their corresponding
  regions, ensuring faithful appearance consistency. Extensive experiments on
  public benchmarks demonstrate the effectiveness of the proposed DGAD framework.
paper_ai_summary: The Disentangled Geometry-editable and Appearance-preserving Diffusion (DGAD) model effectively integrates target objects into background scenes by using semantic embeddings for geometry and cross-attention for appearance alignment.
paper_ai_keywords: disentangled geometry-editable appearance-preserving diffusion diffusion models cross-attention retrieval CLIP/DINO reference networks semantic embeddings appearance-preserving representations flexible object manipulation spatial reasoning capabilities dense cross-attention mechanism public benchmarks