title: Region-Aware Text-to-Image Generation via Hard Binding and Soft
    Refinement
summary: In this paper, we present RAG, a Regional-Aware text-to-image Generation
  method conditioned on regional descriptions for precise layout composition.
  Regional prompting, or compositional generation, which enables fine-grained
  spatial control, has gained increasing attention for its practicality in
  real-world applications. However, previous methods either introduce additional
  trainable modules, thus only applicable to specific models, or manipulate on
  score maps within cross-attention layers using attention masks, resulting in
  limited control strength when the number of regions increases. To handle these
  limitations, we decouple the multi-region generation into two sub-tasks, the
  construction of individual region (Regional Hard Binding) that ensures the
  regional prompt is properly executed, and the overall detail refinement
  (Regional Soft Refinement) over regions that dismiss the visual boundaries and
  enhance adjacent interactions. Furthermore, RAG novelly makes repainting
  feasible, where users can modify specific unsatisfied regions in the last
  generation while keeping all other regions unchanged, without relying on
  additional inpainting models. Our approach is tuning-free and applicable to
  other frameworks as an enhancement to the prompt following property.
  Quantitative and qualitative experiments demonstrate that RAG achieves superior
  performance over attribute binding and object relationship than previous
  tuning-free methods.
paper_authors._id: 6738f5e35604c3c005194615
paper_authors.name: Zhennan Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T16:17:22.954Z
paper_authors._id: 66449e619ff401732687f013
paper_authors.avatarUrl: /avatars/251897d1324a70a9bf761513871c5841.svg
paper_authors.fullname: chen
paper_authors.type: user
paper_authors.user: zhen-nan
paper_authors._id: 6738f5e35604c3c005194616
paper_authors.name: Yajie Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-16T19:45:14.424Z
paper_authors._id: 6669a6e593194f6b71bfa4ef
paper_authors.avatarUrl: /avatars/373c5de91c25f2cb5f2ff8a4e27df3d5.svg
paper_authors.fullname: YajieLi
paper_authors.type: user
paper_authors.user: YajieLife
paper_authors._id: 6738f5e35604c3c005194617
paper_authors.name: Haofan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-16T19:45:21.938Z
paper_authors._id: 637745113a63a2983ffbde13
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669187672174-637745113a63a2983ffbde13.jpeg
paper_authors.fullname: Haofan Wang
paper_authors.type: user
paper_authors.user: wanghaofan
paper_authors._id: 6738f5e35604c3c005194618
paper_authors.name: Zhibo Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-16T19:46:09.076Z
paper_authors._id: 66d963e52e82d53d3b81031b
paper_authors.avatarUrl: /avatars/302dbffc033ff47813a2435a2cec02f1.svg
paper_authors.fullname: Zhibo Chen
paper_authors.type: user
paper_authors.user: winhelp
paper_authors._id: 6738f5e35604c3c005194619
paper_authors.name: Zhengkai Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-18T16:17:32.848Z
paper_authors._id: 6371dda2ffc0489ed7d712c2
paper_authors.avatarUrl: /avatars/dcb80cae00535c22af99baf86cb70d93.svg
paper_authors.fullname: zkjiang
paper_authors.type: user
paper_authors.user: justin-zk
paper_authors._id: 6738f5e35604c3c00519461a
paper_authors.name: Jun Li
paper_authors._id: 6738f5e35604c3c00519461b
paper_authors.name: Qian Wang
paper_authors._id: 6738f5e35604c3c00519461c
paper_authors.name: Jian Yang
paper_authors._id: 6738f5e35604c3c00519461d
paper_authors.name: Ying Tai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-16T19:47:58.217Z
paper_authors._id: 65734004769f3ee9bde1af10
paper_authors.avatarUrl: /avatars/d6310ed861972fd691687d8f47413f33.svg
paper_authors.fullname: Ying Tai
paper_authors.type: user
paper_authors.user: yingtai
paper_title: Region-Aware Text-to-Image Generation via Hard Binding and Soft
    Refinement
paper_summary: In this paper, we present RAG, a Regional-Aware text-to-image Generation
  method conditioned on regional descriptions for precise layout composition.
  Regional prompting, or compositional generation, which enables fine-grained
  spatial control, has gained increasing attention for its practicality in
  real-world applications. However, previous methods either introduce additional
  trainable modules, thus only applicable to specific models, or manipulate on
  score maps within cross-attention layers using attention masks, resulting in
  limited control strength when the number of regions increases. To handle these
  limitations, we decouple the multi-region generation into two sub-tasks, the
  construction of individual region (Regional Hard Binding) that ensures the
  regional prompt is properly executed, and the overall detail refinement
  (Regional Soft Refinement) over regions that dismiss the visual boundaries and
  enhance adjacent interactions. Furthermore, RAG novelly makes repainting
  feasible, where users can modify specific unsatisfied regions in the last
  generation while keeping all other regions unchanged, without relying on
  additional inpainting models. Our approach is tuning-free and applicable to
  other frameworks as an enhancement to the prompt following property.
  Quantitative and qualitative experiments demonstrate that RAG achieves superior
  performance over attribute binding and object relationship than previous
  tuning-free methods.
paper_ai_keywords: Adversarial-Aware text-to-image Generation compositional generation regional prompting cross-attention layers Regional Hard Binding Regional Soft Refinement repainting attribute binding object relationship