title: Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
summary: Diffusion models create data from noise by inverting the forward paths of
  data towards noise and have emerged as a powerful generative modeling technique
  for high-dimensional, perceptual data such as images and videos. Rectified flow
  is a recent generative model formulation that connects data and noise in a
  straight line. Despite its better theoretical properties and conceptual
  simplicity, it is not yet decisively established as standard practice. In this
  work, we improve existing noise sampling techniques for training rectified flow
  models by biasing them towards perceptually relevant scales. Through a
  large-scale study, we demonstrate the superior performance of this approach
  compared to established diffusion formulations for high-resolution
  text-to-image synthesis. Additionally, we present a novel transformer-based
  architecture for text-to-image generation that uses separate weights for the
  two modalities and enables a bidirectional flow of information between image
  and text tokens, improving text comprehension, typography, and human preference
  ratings. We demonstrate that this architecture follows predictable scaling
  trends and correlates lower validation loss to improved text-to-image synthesis
  as measured by various metrics and human evaluations. Our largest models
  outperform state-of-the-art models, and we will make our experimental data,
  code, and model weights publicly available.
paper_authors._id: 65e7d25c9e860e781b24f11f
paper_authors.name: Patrick Esser
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:40:28.177Z
paper_authors._id: 62f2576394fe062ca9adf3d5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660049160797-noauth.png
paper_authors.fullname: Patrick Esser
paper_authors.type: user
paper_authors.user: pesser
paper_authors._id: 65e7d25c9e860e781b24f120
paper_authors.name: Sumith Kulal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:40:41.408Z
paper_authors._id: 630fd26be52a259b856600d3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630fd26be52a259b856600d3/yAmMYc35ZDLDbE5Hpurs1.jpeg
paper_authors.fullname: Sumith Kulal
paper_authors.type: user
paper_authors.user: sumith1896
paper_authors._id: 65e7d25c9e860e781b24f121
paper_authors.name: Andreas Blattmann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:40:56.954Z
paper_authors._id: 635fbc957c05eb9f5997b568
paper_authors.avatarUrl: /avatars/0bf85960fa7ad445b0abb2d70c9d5721.svg
paper_authors.fullname: Andreas Blattmann
paper_authors.type: user
paper_authors.user: ablattmann
paper_authors._id: 65e7d25c9e860e781b24f122
paper_authors.name: Rahim Entezari
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:41:04.031Z
paper_authors._id: 62fca7e7b17dbe0ceae54f04
paper_authors.avatarUrl: /avatars/90c55bd5e0c6842586cc0ec5db387f89.svg
paper_authors.fullname: Rahim Entezari
paper_authors.type: user
paper_authors.user: rahiment
paper_authors._id: 65e7d25c9e860e781b24f123
paper_authors.name: Jonas Müller
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:41:29.350Z
paper_authors._id: 64510d6304397681bcf9725b
paper_authors.avatarUrl: /avatars/acb0f7305807539f892472ef4e847310.svg
paper_authors.fullname: Jonas Müller
paper_authors.type: user
paper_authors.user: jaymueller
paper_authors._id: 65e7d25c9e860e781b24f124
paper_authors.name: Harry Saini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:41:37.445Z
paper_authors._id: 619252d5ded356549e20c5cd
paper_authors.avatarUrl: /avatars/c3ea01b1054bbdfa891db94aa39e745c.svg
paper_authors.fullname: Harry 
paper_authors.type: user
paper_authors.user: HarrySaini
paper_authors._id: 65e7d25c9e860e781b24f125
paper_authors.name: Yam Levi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:41:43.643Z
paper_authors._id: 64df2032842e7056589bc43e
paper_authors.avatarUrl: /avatars/7cae8e7cf685b1368cc66ff981927522.svg
paper_authors.fullname: Yam Levi
paper_authors.type: user
paper_authors.user: yumlevi
paper_authors._id: 65e7d25c9e860e781b24f126
paper_authors.name: Dominik Lorenz
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:41:56.184Z
paper_authors._id: 64c0fd7c36db3901addfd28a
paper_authors.avatarUrl: /avatars/8ff6247de6d5906e0e5431edbdf943ca.svg
paper_authors.fullname: Dominik Lorenz
paper_authors.type: user
paper_authors.user: dominiklorenz
paper_authors._id: 65e7d25c9e860e781b24f127
paper_authors.name: Axel Sauer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:42:06.543Z
paper_authors._id: 61fc209cef99814f1705e934
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61fc209cef99814f1705e934/PSgCbKtcCb5dg84513fo-.jpeg
paper_authors.fullname: Axel Sauer
paper_authors.type: user
paper_authors.user: xl-sr
paper_authors._id: 65e7d25c9e860e781b24f128
paper_authors.name: Frederic Boesel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:42:12.418Z
paper_authors._id: 653beb48ccfb7ba1cbbc9218
paper_authors.avatarUrl: /avatars/f14a52db958f56d3082c2545b88ea99a.svg
paper_authors.fullname: Frederic Boesel
paper_authors.type: user
paper_authors.user: fboeEl
paper_authors._id: 65e7d25c9e860e781b24f129
paper_authors.name: Dustin Podell
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:42:25.480Z
paper_authors._id: 63055631acc17ce4ad369a37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676933320376-63055631acc17ce4ad369a37.png
paper_authors.fullname: Dustin Podell
paper_authors.type: user
paper_authors.user: TwoDukes
paper_authors._id: 65e7d25c9e860e781b24f12a
paper_authors.name: Tim Dockhorn
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:42:32.066Z
paper_authors._id: 633f7a8f4be90e06da248e0f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633f7a8f4be90e06da248e0f/Kf-vMGCSIar8UGezk-TK9.jpeg
paper_authors.fullname: Tim Dockhorn
paper_authors.type: user
paper_authors.user: timudk
paper_authors._id: 65e7d25c9e860e781b24f12b
paper_authors.name: Zion English
paper_authors._id: 65e7d25c9e860e781b24f12c
paper_authors.name: Kyle Lacey
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:42:42.635Z
paper_authors._id: 6318c908acffc70bd4deebec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6318c908acffc70bd4deebec/2I88otvT1rDot_22yYPIi.png
paper_authors.fullname: TwoPerCent
paper_authors.type: user
paper_authors.user: TwoPerCent
paper_authors._id: 65e7d25c9e860e781b24f12d
paper_authors.name: Alex Goodwin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:42:49.028Z
paper_authors._id: 64c12d35cac5f9ba52b68720
paper_authors.avatarUrl: /avatars/d0c2709aa77c98b4401f0ae4bda68e04.svg
paper_authors.fullname: Alex Goodwin
paper_authors.type: user
paper_authors.user: AlexGoodwin
paper_authors._id: 65e7d25c9e860e781b24f12e
paper_authors.name: Yannik Marek
paper_authors._id: 65e7d25c9e860e781b24f12f
paper_authors.name: Robin Rombach
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T09:42:58.961Z
paper_authors._id: 62cfefa74b3e8dc1e32e38bf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1657794439283-noauth.png
paper_authors.fullname: Robin Rombach
paper_authors.type: user
paper_authors.user: rromb
paper_title: Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
paper_summary: Diffusion models create data from noise by inverting the forward paths of
  data towards noise and have emerged as a powerful generative modeling technique
  for high-dimensional, perceptual data such as images and videos. Rectified flow
  is a recent generative model formulation that connects data and noise in a
  straight line. Despite its better theoretical properties and conceptual
  simplicity, it is not yet decisively established as standard practice. In this
  work, we improve existing noise sampling techniques for training rectified flow
  models by biasing them towards perceptually relevant scales. Through a
  large-scale study, we demonstrate the superior performance of this approach
  compared to established diffusion formulations for high-resolution
  text-to-image synthesis. Additionally, we present a novel transformer-based
  architecture for text-to-image generation that uses separate weights for the
  two modalities and enables a bidirectional flow of information between image
  and text tokens, improving text comprehension, typography, and human preference
  ratings. We demonstrate that this architecture follows predictable scaling
  trends and correlates lower validation loss to improved text-to-image synthesis
  as measured by various metrics and human evaluations. Our largest models
  outperform state-of-the-art models, and we will make our experimental data,
  code, and model weights publicly available.
paper_ai_keywords: diffusion models rectified flow noise sampling techniques perceptually relevant scales high-resolution text-to-image synthesis transformer-based architecture bidirectional flow image and text tokens text comprehension typography human preference ratings validation loss text-to-image synthesis