title: DART: Denoising Autoregressive Transformer for Scalable Text-to-Image
    Generation
summary: Diffusion models have become the dominant approach for visual generation.
  They are trained by denoising a Markovian process that gradually adds noise to
  the input. We argue that the Markovian property limits the models ability to
  fully utilize the generation trajectory, leading to inefficiencies during
  training and inference. In this paper, we propose DART, a transformer-based
  model that unifies autoregressive (AR) and diffusion within a non-Markovian
  framework. DART iteratively denoises image patches spatially and spectrally
  using an AR model with the same architecture as standard language models. DART
  does not rely on image quantization, enabling more effective image modeling
  while maintaining flexibility. Furthermore, DART seamlessly trains with both
  text and image data in a unified model. Our approach demonstrates competitive
  performance on class-conditioned and text-to-image generation tasks, offering a
  scalable, efficient alternative to traditional diffusion models. Through this
  unified framework, DART sets a new benchmark for scalable, high-quality image
  synthesis.
paper_authors._id: 67093e42f1cc665b4ca86129
paper_authors.name: Jiatao Gu
paper_authors._id: 67093e42f1cc665b4ca8612a
paper_authors.name: Yuyang Wang
paper_authors._id: 67093e42f1cc665b4ca8612b
paper_authors.name: Yizhe Zhang
paper_authors._id: 67093e42f1cc665b4ca8612c
paper_authors.name: Qihang Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T07:48:37.496Z
paper_authors._id: 63047acc41387c7f11756284
paper_authors.avatarUrl: /avatars/a184fca1ee2d98b81e1917aecc3cdca7.svg
paper_authors.fullname: i3h
paper_authors.type: user
paper_authors.user: qihang
paper_authors._id: 67093e42f1cc665b4ca8612d
paper_authors.name: Dinghuai Zhang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-10-11T15:03:35.154Z
paper_authors._id: 64384e18d221ff12edae4c75
paper_authors.avatarUrl: /avatars/3a5d2400af0f26091c233d63984df412.svg
paper_authors.fullname: Dinghuai Zhang
paper_authors.type: user
paper_authors.user: Dinghuai
paper_authors._id: 67093e42f1cc665b4ca8612e
paper_authors.name: Navdeep Jaitly
paper_authors._id: 67093e42f1cc665b4ca8612f
paper_authors.name: Josh Susskind
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-22T16:29:37.453Z
paper_authors._id: 6470d2247fd7ecdbd0ec3cc9
paper_authors.avatarUrl: /avatars/52c5eca12499a1aa9bd49c43d4f20685.svg
paper_authors.fullname: Joshua M. Susskind
paper_authors.type: user
paper_authors.user: jsusskind
paper_authors._id: 67093e42f1cc665b4ca86130
paper_authors.name: Shuangfei Zhai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-13T15:23:32.031Z
paper_authors._id: 675bf04240726aac79ae8e7b
paper_authors.avatarUrl: /avatars/8ae95255dfce91a6f5e76f14eae9e9e5.svg
paper_authors.fullname: Shuangfei Zhai
paper_authors.type: user
paper_authors.user: shuangfei
paper_title: DART: Denoising Autoregressive Transformer for Scalable Text-to-Image
    Generation
paper_summary: Diffusion models have become the dominant approach for visual generation.
  They are trained by denoising a Markovian process that gradually adds noise to
  the input. We argue that the Markovian property limits the models ability to
  fully utilize the generation trajectory, leading to inefficiencies during
  training and inference. In this paper, we propose DART, a transformer-based
  model that unifies autoregressive (AR) and diffusion within a non-Markovian
  framework. DART iteratively denoises image patches spatially and spectrally
  using an AR model with the same architecture as standard language models. DART
  does not rely on image quantization, enabling more effective image modeling
  while maintaining flexibility. Furthermore, DART seamlessly trains with both
  text and image data in a unified model. Our approach demonstrates competitive
  performance on class-conditioned and text-to-image generation tasks, offering a
  scalable, efficient alternative to traditional diffusion models. Through this
  unified framework, DART sets a new benchmark for scalable, high-quality image
  synthesis.
paper_ai_summary: DART, a transformer-based model combining autoregressive and diffusion components in a non-Markovian framework, offers competitive performance on image and text-to-image generation tasks.
paper_ai_keywords: diffusion models Markovian process autoregressive (AR) models non-Markovian framework image patches spectral denoising image quantization class-conditioned generation text-to-image generation