title: DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale
    Text-to-Image Models
summary: Image-conditioned generation methods, such as depth- and canny-conditioned
  approaches, have demonstrated remarkable abilities for precise image synthesis.
  However, existing models still struggle to accurately control the content of
  multiple instances (or regions). Even state-of-the-art models like FLUX and
  3DIS face challenges, such as attribute leakage between instances, which limits
  user control. To address these issues, we introduce DreamRenderer, a
  training-free approach built upon the FLUX model. DreamRenderer enables users
  to control the content of each instance via bounding boxes or masks, while
  ensuring overall visual harmony. We propose two key innovations: 1) Bridge
  Image Tokens for Hard Text Attribute Binding, which uses replicated image
  tokens as bridge tokens to ensure that T5 text embeddings, pre-trained solely
  on text data, bind the correct visual attributes for each instance during Joint
  Attention; 2) Hard Image Attribute Binding applied only to vital layers.
  Through our analysis of FLUX, we identify the critical layers responsible for
  instance attribute rendering and apply Hard Image Attribute Binding only in
  these layers, using soft binding in the others. This approach ensures precise
  control while preserving image quality. Evaluations on the COCO-POS and
  COCO-MIG benchmarks demonstrate that DreamRenderer improves the Image Success
  Ratio by 17.7% over FLUX and enhances the performance of layout-to-image models
  like GLIGEN and 3DIS by up to 26.8%. Project Page:
  https://limuloo.github.io/DreamRenderer/.
paper_authors._id: 67d8e23afa59a8b15a9057e8
paper_authors.name: Dewei Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-18T09:20:39.038Z
paper_authors._id: 65eaa1e2b11eeb516a973508
paper_authors.avatarUrl: /avatars/beecd135bb940fdc02406f9063b3fa67.svg
paper_authors.fullname: Dewei Zhou
paper_authors.type: user
paper_authors.user: limuloo1999
paper_authors._id: 67d8e23afa59a8b15a9057e9
paper_authors.name: Mingwei Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-18T09:20:46.810Z
paper_authors._id: 64551bc2c9c0dcc8c2484cf6
paper_authors.avatarUrl: /avatars/0d1ed4f4502f6f54ac6ba071e4c9a220.svg
paper_authors.fullname: Mingwei Li
paper_authors.type: user
paper_authors.user: aiJojosh
paper_authors._id: 67d8e23afa59a8b15a9057ea
paper_authors.name: Zongxin Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-18T09:20:52.869Z
paper_authors._id: 619bf9b3cbedb87e1a92fb3b
paper_authors.avatarUrl: /avatars/ee280db0232e21416c948ab9a9a2344e.svg
paper_authors.fullname: Zongxin Yang
paper_authors.type: user
paper_authors.user: z-x-yang
paper_authors._id: 67d8e23afa59a8b15a9057eb
paper_authors.name: Yi Yang
paper_title: DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale
    Text-to-Image Models
paper_summary: Image-conditioned generation methods, such as depth- and canny-conditioned
  approaches, have demonstrated remarkable abilities for precise image synthesis.
  However, existing models still struggle to accurately control the content of
  multiple instances (or regions). Even state-of-the-art models like FLUX and
  3DIS face challenges, such as attribute leakage between instances, which limits
  user control. To address these issues, we introduce DreamRenderer, a
  training-free approach built upon the FLUX model. DreamRenderer enables users
  to control the content of each instance via bounding boxes or masks, while
  ensuring overall visual harmony. We propose two key innovations: 1) Bridge
  Image Tokens for Hard Text Attribute Binding, which uses replicated image
  tokens as bridge tokens to ensure that T5 text embeddings, pre-trained solely
  on text data, bind the correct visual attributes for each instance during Joint
  Attention; 2) Hard Image Attribute Binding applied only to vital layers.
  Through our analysis of FLUX, we identify the critical layers responsible for
  instance attribute rendering and apply Hard Image Attribute Binding only in
  these layers, using soft binding in the others. This approach ensures precise
  control while preserving image quality. Evaluations on the COCO-POS and
  COCO-MIG benchmarks demonstrate that DreamRenderer improves the Image Success
  Ratio by 17.7% over FLUX and enhances the performance of layout-to-image models
  like GLIGEN and 3DIS by up to 26.8%. Project Page:
  https://limuloo.github.io/DreamRenderer/.
paper_ai_summary: DreamRenderer enhances image-conditioned generation by precisely controlling instance attributes using bounding boxes and masks, improving the Image Success Ratio and layout-to-image performance.
paper_ai_keywords: depth-conditioned canny-conditioned attribute leakage DreamRenderer Bridge Image Tokens T5 text embeddings Joint Attention Hard Image Attribute Binding COCO-POS COCO-MIG Image Success Ratio GLIGEN 3DIS