DART: Denoising Autoregressive Transformer for Scalable Text-to-Image
  Generation
Diffusion models have become the dominant approach for visual generation.
They are trained by denoising a Markovian process that gradually adds noise to
the input. We argue that the Markovian property limits the models ability to
fully utilize the generation trajectory, leading to inefficiencies during
training and inference. In this paper, we propose DART, a transformer-based
model that unifies autoregressive (AR) and diffusion within a non-Markovian
framework. DART iteratively denoises image patches spatially and spectrally
using an AR model with the same architecture as standard language models. DART
does not rely on image quantization, enabling more effective image modeling
while maintaining flexibility. Furthermore, DART seamlessly trains with both
text and image data in a unified model. Our approach demonstrates competitive
performance on class-conditioned and text-to-image generation tasks, offering a
scalable, efficient alternative to traditional diffusion models. Through this
unified framework, DART sets a new benchmark for scalable, high-quality image
synthesis.
67093e42f1cc665b4ca86129
Jiatao Gu
67093e42f1cc665b4ca8612a
Yuyang Wang
67093e42f1cc665b4ca8612b
Yizhe Zhang
67093e42f1cc665b4ca8612c
Qihang Zhang
claimed_verified
2024-10-14T07:48:37.496Z
63047acc41387c7f11756284
/avatars/a184fca1ee2d98b81e1917aecc3cdca7.svg
i3h
user
qihang
67093e42f1cc665b4ca8612d
Dinghuai Zhang
extracted_pending
2024-10-11T15:03:35.154Z
64384e18d221ff12edae4c75
/avatars/3a5d2400af0f26091c233d63984df412.svg
Dinghuai Zhang
user
Dinghuai
67093e42f1cc665b4ca8612e
Navdeep Jaitly
67093e42f1cc665b4ca8612f
Josh Susskind
claimed_verified
2024-11-22T16:29:37.453Z
6470d2247fd7ecdbd0ec3cc9
/avatars/52c5eca12499a1aa9bd49c43d4f20685.svg
Joshua M. Susskind
user
jsusskind
67093e42f1cc665b4ca86130
Shuangfei Zhai
claimed_verified
2024-12-13T15:23:32.031Z
675bf04240726aac79ae8e7b
/avatars/8ae95255dfce91a6f5e76f14eae9e9e5.svg
Shuangfei Zhai
user
shuangfei
DART: Denoising Autoregressive Transformer for Scalable Text-to-Image
  Generation
Diffusion models have become the dominant approach for visual generation.
They are trained by denoising a Markovian process that gradually adds noise to
the input. We argue that the Markovian property limits the models ability to
fully utilize the generation trajectory, leading to inefficiencies during
training and inference. In this paper, we propose DART, a transformer-based
model that unifies autoregressive (AR) and diffusion within a non-Markovian
framework. DART iteratively denoises image patches spatially and spectrally
using an AR model with the same architecture as standard language models. DART
does not rely on image quantization, enabling more effective image modeling
while maintaining flexibility. Furthermore, DART seamlessly trains with both
text and image data in a unified model. Our approach demonstrates competitive
performance on class-conditioned and text-to-image generation tasks, offering a
scalable, efficient alternative to traditional diffusion models. Through this
unified framework, DART sets a new benchmark for scalable, high-quality image
synthesis.
DART, a transformer-based model combining autoregressive and diffusion components in a non-Markovian framework, offers competitive performance on image and text-to-image generation tasks.
diffusion models Markovian process autoregressive (AR) models non-Markovian framework image patches spectral denoising image quantization class-conditioned generation text-to-image generation