Region-Aware Text-to-Image Generation via Hard Binding and Soft
  Refinement
In this paper, we present RAG, a Regional-Aware text-to-image Generation
method conditioned on regional descriptions for precise layout composition.
Regional prompting, or compositional generation, which enables fine-grained
spatial control, has gained increasing attention for its practicality in
real-world applications. However, previous methods either introduce additional
trainable modules, thus only applicable to specific models, or manipulate on
score maps within cross-attention layers using attention masks, resulting in
limited control strength when the number of regions increases. To handle these
limitations, we decouple the multi-region generation into two sub-tasks, the
construction of individual region (Regional Hard Binding) that ensures the
regional prompt is properly executed, and the overall detail refinement
(Regional Soft Refinement) over regions that dismiss the visual boundaries and
enhance adjacent interactions. Furthermore, RAG novelly makes repainting
feasible, where users can modify specific unsatisfied regions in the last
generation while keeping all other regions unchanged, without relying on
additional inpainting models. Our approach is tuning-free and applicable to
other frameworks as an enhancement to the prompt following property.
Quantitative and qualitative experiments demonstrate that RAG achieves superior
performance over attribute binding and object relationship than previous
tuning-free methods.
6738f5e35604c3c005194615
Zhennan Chen
admin_assigned
2024-11-18T16:17:22.954Z
66449e619ff401732687f013
/avatars/251897d1324a70a9bf761513871c5841.svg
chen
user
zhen-nan
6738f5e35604c3c005194616
Yajie Li
admin_assigned
2024-11-16T19:45:14.424Z
6669a6e593194f6b71bfa4ef
/avatars/373c5de91c25f2cb5f2ff8a4e27df3d5.svg
YajieLi
user
YajieLife
6738f5e35604c3c005194617
Haofan Wang
admin_assigned
2024-11-16T19:45:21.938Z
637745113a63a2983ffbde13
https://cdn-avatars.huggingface.co/v1/production/uploads/1669187672174-637745113a63a2983ffbde13.jpeg
Haofan Wang
user
wanghaofan
6738f5e35604c3c005194618
Zhibo Chen
admin_assigned
2024-11-16T19:46:09.076Z
66d963e52e82d53d3b81031b
/avatars/302dbffc033ff47813a2435a2cec02f1.svg
Zhibo Chen
user
winhelp
6738f5e35604c3c005194619
Zhengkai Jiang
claimed_verified
2024-11-18T16:17:32.848Z
6371dda2ffc0489ed7d712c2
/avatars/dcb80cae00535c22af99baf86cb70d93.svg
zkjiang
user
justin-zk
6738f5e35604c3c00519461a
Jun Li
6738f5e35604c3c00519461b
Qian Wang
6738f5e35604c3c00519461c
Jian Yang
6738f5e35604c3c00519461d
Ying Tai
admin_assigned
2024-11-16T19:47:58.217Z
65734004769f3ee9bde1af10
/avatars/d6310ed861972fd691687d8f47413f33.svg
Ying Tai
user
yingtai
Region-Aware Text-to-Image Generation via Hard Binding and Soft
  Refinement
In this paper, we present RAG, a Regional-Aware text-to-image Generation
method conditioned on regional descriptions for precise layout composition.
Regional prompting, or compositional generation, which enables fine-grained
spatial control, has gained increasing attention for its practicality in
real-world applications. However, previous methods either introduce additional
trainable modules, thus only applicable to specific models, or manipulate on
score maps within cross-attention layers using attention masks, resulting in
limited control strength when the number of regions increases. To handle these
limitations, we decouple the multi-region generation into two sub-tasks, the
construction of individual region (Regional Hard Binding) that ensures the
regional prompt is properly executed, and the overall detail refinement
(Regional Soft Refinement) over regions that dismiss the visual boundaries and
enhance adjacent interactions. Furthermore, RAG novelly makes repainting
feasible, where users can modify specific unsatisfied regions in the last
generation while keeping all other regions unchanged, without relying on
additional inpainting models. Our approach is tuning-free and applicable to
other frameworks as an enhancement to the prompt following property.
Quantitative and qualitative experiments demonstrate that RAG achieves superior
performance over attribute binding and object relationship than previous
tuning-free methods.
RAG, a Regional-Aware text-to-image generation method, achieves fine-grained spatial control by decoupling generation into two sub-tasks, ensuring robust performance in real-world applications.
regional-aware generation regional prompting compositional generation fine-grained spatial control regional hard binding regional soft refinement repainting