title: SiT: Exploring Flow and Diffusion-based Generative Models with Scalable
    Interpolant Transformers
summary: We present Scalable Interpolant Transformers (SiT), a family of generative
  models built on the backbone of Diffusion Transformers (DiT). The interpolant
  framework, which allows for connecting two distributions in a more flexible way
  than standard diffusion models, makes possible a modular study of various
  design choices impacting generative models built on dynamical transport: using
  discrete vs. continuous time learning, deciding the objective for the model to
  learn, choosing the interpolant connecting the distributions, and deploying a
  deterministic or stochastic sampler. By carefully introducing the above
  ingredients, SiT surpasses DiT uniformly across model sizes on the conditional
  ImageNet 256x256 benchmark using the exact same backbone, number of parameters,
  and GFLOPs. By exploring various diffusion coefficients, which can be tuned
  separately from learning, SiT achieves an FID-50K score of 2.06.
paper_authors._id: 65a8afd76cb12bdc272f2774
paper_authors.name: Nanye Ma
paper_authors._id: 65a8afd76cb12bdc272f2775
paper_authors.name: Mark Goldstein
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-18T09:57:45.428Z
paper_authors._id: 6470f300d9360cd9d8e9219b
paper_authors.avatarUrl: /avatars/dc09260e9912d7e626cad5c670ec8d3f.svg
paper_authors.fullname: mark goldstein
paper_authors.type: user
paper_authors.user: marikgoldstein
paper_authors._id: 65a8afd76cb12bdc272f2776
paper_authors.name: Michael S. Albergo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-18T09:58:12.428Z
paper_authors._id: 636fbc0731af06da864cd7eb
paper_authors.avatarUrl: /avatars/415db2ff8f79440a0216dd39cf6d0d43.svg
paper_authors.fullname: Michael Albergo
paper_authors.type: user
paper_authors.user: malbergo
paper_authors._id: 65a8afd76cb12bdc272f2777
paper_authors.name: Nicholas M. Boffi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-23T17:09:22.554Z
paper_authors._id: 65afd8b7bd418aab3a677782
paper_authors.avatarUrl: /avatars/11aa33b32ef4b6aa27ab8e6c068afa14.svg
paper_authors.fullname: Nicholas Boffi
paper_authors.type: user
paper_authors.user: nmboffi
paper_authors._id: 65a8afd76cb12bdc272f2778
paper_authors.name: Eric Vanden-Eijnden
paper_authors._id: 65a8afd76cb12bdc272f2779
paper_authors.name: Saining Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-18T09:58:57.348Z
paper_authors._id: 6596422646624a86ff3b3bda
paper_authors.avatarUrl: /avatars/216e12b77e45ac5f1fa20932f5745411.svg
paper_authors.fullname: Saining Xie
paper_authors.type: user
paper_authors.user: sainx
paper_title: SiT: Exploring Flow and Diffusion-based Generative Models with Scalable
    Interpolant Transformers
paper_summary: We present Scalable Interpolant Transformers (SiT), a family of generative
  models built on the backbone of Diffusion Transformers (DiT). The interpolant
  framework, which allows for connecting two distributions in a more flexible way
  than standard diffusion models, makes possible a modular study of various
  design choices impacting generative models built on dynamical transport: using
  discrete vs. continuous time learning, deciding the objective for the model to
  learn, choosing the interpolant connecting the distributions, and deploying a
  deterministic or stochastic sampler. By carefully introducing the above
  ingredients, SiT surpasses DiT uniformly across model sizes on the conditional
  ImageNet 256x256 benchmark using the exact same backbone, number of parameters,
  and GFLOPs. By exploring various diffusion coefficients, which can be tuned
  separately from learning, SiT achieves an FID-50K score of 2.06.
paper_ai_summary: Scalable Interpolant Transformers (SiT), built on Diffusion Transformers (DiT), enhance generative models through a flexible interpolant framework, achieving superior performance on ImageNet 256x256 with tunable diffusion coefficients.
paper_ai_keywords: Scalable Interpolant Transformers SiT Diffusion Transformers DiT interpolant framework dynamical transport discrete time learning continuous time learning stochastic sampler diffusion coefficients FID-50K score