title: Training-free Regional Prompting for Diffusion Transformers
summary: Diffusion models have demonstrated excellent capabilities in text-to-image
  generation. Their semantic understanding (i.e., prompt following) ability has
  also been greatly improved with large language models (e.g., T5, Llama).
  However, existing models cannot perfectly handle long and complex text prompts,
  especially when the text prompts contain various objects with numerous
  attributes and interrelated spatial relationships. While many regional
  prompting methods have been proposed for UNet-based models (SD1.5, SDXL), but
  there are still no implementations based on the recent Diffusion Transformer
  (DiT) architecture, such as SD3 and FLUX.1.In this report, we propose and
  implement regional prompting for FLUX.1 based on attention manipulation, which
  enables DiT with fined-grained compositional text-to-image generation
  capability in a training-free manner. Code is available at
  https://github.com/antonioo-c/Regional-Prompting-FLUX.
paper_authors._id: 6729ba9886503ffe1cc31897
paper_authors.name: Anthony Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-05T07:57:18.751Z
paper_authors._id: 6311d9ee04f842f79916158c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/PvOYgDEpe0x5ExLyj1BK-.png
paper_authors.fullname: chen
paper_authors.type: user
paper_authors.user: antonio-c
paper_authors._id: 6729ba9886503ffe1cc31898
paper_authors.name: Jianjin Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-05T13:59:09.353Z
paper_authors._id: 6323c5a899c4839e1c6223e9
paper_authors.avatarUrl: /avatars/7890aa86a309e2c86e7341212f8afc40.svg
paper_authors.fullname: Jianjin Xu
paper_authors.type: user
paper_authors.user: atlantix
paper_authors._id: 6729ba9886503ffe1cc31899
paper_authors.name: Wenzhao Zheng
paper_authors._id: 6729ba9886503ffe1cc3189a
paper_authors.name: Gaole Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-05T13:59:23.394Z
paper_authors._id: 668d9de515511820b9c31a51
paper_authors.avatarUrl: /avatars/409f1334969f659a331a61bb0f4ce15c.svg
paper_authors.fullname: Gaole Dai
paper_authors.type: user
paper_authors.user: barkleydai
paper_authors._id: 6729ba9886503ffe1cc3189b
paper_authors.name: Yida Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-05T07:57:13.829Z
paper_authors._id: 647068944be5cf1f33491cb9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Aqudl2PTSKPAylBhlccWr.png
paper_authors.fullname: Yida Wang
paper_authors.type: user
paper_authors.user: wangyida
paper_authors._id: 6729ba9886503ffe1cc3189c
paper_authors.name: Renrui Zhang
paper_authors._id: 6729ba9886503ffe1cc3189d
paper_authors.name: Haofan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-05T07:57:15.554Z
paper_authors._id: 637745113a63a2983ffbde13
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669187672174-637745113a63a2983ffbde13.jpeg
paper_authors.fullname: Haofan Wang
paper_authors.type: user
paper_authors.user: wanghaofan
paper_authors._id: 6729ba9886503ffe1cc3189e
paper_authors.name: Shanghang Zhang
paper_title: Training-free Regional Prompting for Diffusion Transformers
paper_summary: Diffusion models have demonstrated excellent capabilities in text-to-image
  generation. Their semantic understanding (i.e., prompt following) ability has
  also been greatly improved with large language models (e.g., T5, Llama).
  However, existing models cannot perfectly handle long and complex text prompts,
  especially when the text prompts contain various objects with numerous
  attributes and interrelated spatial relationships. While many regional
  prompting methods have been proposed for UNet-based models (SD1.5, SDXL), but
  there are still no implementations based on the recent Diffusion Transformer
  (DiT) architecture, such as SD3 and FLUX.1.In this report, we propose and
  implement regional prompting for FLUX.1 based on attention manipulation, which
  enables DiT with fined-grained compositional text-to-image generation
  capability in a training-free manner. Code is available at
  https://github.com/antonioo-c/Regional-Prompting-FLUX.
paper_ai_keywords: diffusion models text-to-image generation semantic understanding prompt following large language models T5 Llama UNet-based models SD1.5 SDXL Diffusion Transformer DiT SD3 FLUX regional prompting attention manipulation fined-grained compositional text-to-image generation training-free manner