title: IterComp: Iterative Composition-Aware Feedback Learning from Model
    Gallery for Text-to-Image Generation
summary: Advanced diffusion models like RPG, Stable Diffusion 3 and FLUX have made
  notable strides in compositional text-to-image generation. However, these
  methods typically exhibit distinct strengths for compositional generation, with
  some excelling in handling attribute binding and others in spatial
  relationships. This disparity highlights the need for an approach that can
  leverage the complementary strengths of various models to comprehensively
  improve the composition capability. To this end, we introduce IterComp, a novel
  framework that aggregates composition-aware model preferences from multiple
  models and employs an iterative feedback learning approach to enhance
  compositional generation. Specifically, we curate a gallery of six powerful
  open-source diffusion models and evaluate their three key compositional
  metrics: attribute binding, spatial relationships, and non-spatial
  relationships. Based on these metrics, we develop a composition-aware model
  preference dataset comprising numerous image-rank pairs to train
  composition-aware reward models. Then, we propose an iterative feedback
  learning method to enhance compositionality in a closed-loop manner, enabling
  the progressive self-refinement of both the base diffusion model and reward
  models over multiple iterations. Theoretical proof demonstrates the
  effectiveness and extensive experiments show our significant superiority over
  previous SOTA methods (e.g., Omost and FLUX), particularly in multi-category
  object composition and complex semantic alignment. IterComp opens new research
  avenues in reward feedback learning for diffusion models and compositional
  generation. Code: https://github.com/YangLing0818/IterComp
paper_authors._id: 67073ff0327cec6882ea2ce1
paper_authors.name: Xinchen Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T08:01:19.383Z
paper_authors._id: 653e5d31ffd60206c8b64bb5
paper_authors.avatarUrl: /avatars/5076795722ec1f9e031654f301d30e8f.svg
paper_authors.fullname: Xinchen Zhang
paper_authors.type: user
paper_authors.user: comin
paper_authors._id: 67073ff0327cec6882ea2ce2
paper_authors.name: Ling Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-11T15:34:27.425Z
paper_authors._id: 64fde4e252e82dd432b74ce9
paper_authors.avatarUrl: /avatars/061a69d858b86d1600be916122cae7fc.svg
paper_authors.fullname: Ling Yang
paper_authors.type: user
paper_authors.user: Lingaaaaaaa
paper_authors._id: 67073ff0327cec6882ea2ce3
paper_authors.name: Guohao Li
paper_authors._id: 67073ff0327cec6882ea2ce4
paper_authors.name: Yaqi Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:44:03.195Z
paper_authors._id: 657fe0b1f010d76b6e709110
paper_authors.avatarUrl: /avatars/9c08817484cfb5b238d6833d788fa0a3.svg
paper_authors.fullname: YaQi Cai
paper_authors.type: user
paper_authors.user: yaqicc
paper_authors._id: 67073ff0327cec6882ea2ce5
paper_authors.name: Jiake Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:43:34.868Z
paper_authors._id: 63f46f4f21eb234ab7395456
paper_authors.avatarUrl: /avatars/16f444058549b0317243784e9e1e3110.svg
paper_authors.fullname: jiake xie
paper_authors.type: user
paper_authors.user: kelisiya
paper_authors._id: 67073ff0327cec6882ea2ce6
paper_authors.name: Yong Tang
paper_authors._id: 67073ff0327cec6882ea2ce7
paper_authors.name: Yujiu Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:43:28.498Z
paper_authors._id: 64ca1fe838837b12d5e529b7
paper_authors.avatarUrl: /avatars/44a3ad9e59318784ac531993b5f69f6b.svg
paper_authors.fullname: Yujiu Yang
paper_authors.type: user
paper_authors.user: Thu-redrobot
paper_authors._id: 67073ff0327cec6882ea2ce8
paper_authors.name: Mengdi Wang
paper_authors._id: 67073ff0327cec6882ea2ce9
paper_authors.name: Bin Cui
paper_title: IterComp: Iterative Composition-Aware Feedback Learning from Model
    Gallery for Text-to-Image Generation
paper_summary: Advanced diffusion models like RPG, Stable Diffusion 3 and FLUX have made
  notable strides in compositional text-to-image generation. However, these
  methods typically exhibit distinct strengths for compositional generation, with
  some excelling in handling attribute binding and others in spatial
  relationships. This disparity highlights the need for an approach that can
  leverage the complementary strengths of various models to comprehensively
  improve the composition capability. To this end, we introduce IterComp, a novel
  framework that aggregates composition-aware model preferences from multiple
  models and employs an iterative feedback learning approach to enhance
  compositional generation. Specifically, we curate a gallery of six powerful
  open-source diffusion models and evaluate their three key compositional
  metrics: attribute binding, spatial relationships, and non-spatial
  relationships. Based on these metrics, we develop a composition-aware model
  preference dataset comprising numerous image-rank pairs to train
  composition-aware reward models. Then, we propose an iterative feedback
  learning method to enhance compositionality in a closed-loop manner, enabling
  the progressive self-refinement of both the base diffusion model and reward
  models over multiple iterations. Theoretical proof demonstrates the
  effectiveness and extensive experiments show our significant superiority over
  previous SOTA methods (e.g., Omost and FLUX), particularly in multi-category
  object composition and complex semantic alignment. IterComp opens new research
  avenues in reward feedback learning for diffusion models and compositional
  generation. Code: https://github.com/YangLing0818/IterComp
paper_ai_keywords: diffusion models RPG Stable Diffusion 3 FLUX compositional text-to-image generation attribute binding spatial relationships non-spatial relationships IterComp composition-aware model preferences iterative feedback learning image-rank pairs composition-aware reward models closed-loop manner multi-category object composition complex semantic alignment reward feedback learning