D-AR: Diffusion via Autoregressive Models
This paper presents Diffusion via Autoregressive models (D-AR), a new
paradigm recasting the image diffusion process as a vanilla autoregressive
procedure in the standard next-token-prediction fashion. We start by designing
the tokenizer that converts images into sequences of discrete tokens, where
tokens in different positions can be decoded into different diffusion denoising
steps in the pixel space. Thanks to the diffusion properties, these tokens
naturally follow a coarse-to-fine order, which directly lends itself to
autoregressive modeling. Therefore, we apply standard next-token prediction on
these tokens, without modifying any underlying designs (either causal masks or
training/inference strategies), and such sequential autoregressive token
generation directly mirrors the diffusion procedure in image space. That is,
once the autoregressive model generates an increment of tokens, we can directly
decode these tokens into the corresponding diffusion denoising step in the
streaming manner. Our pipeline naturally reveals several intriguing properties,
for example, it supports consistent previews when generating only a subset of
tokens and enables zero-shot layout-controlled synthesis. On the standard
ImageNet benchmark, our method achieves 2.09 FID using a 775M Llama backbone
with 256 discrete tokens. We hope our work can inspire future research on
unified autoregressive architectures of visual synthesis, especially with large
language models. Code and models will be available at
https://github.com/showlab/D-AR
683906adf85de1fc563957d8
Ziteng Gao
extracted_pending
2025-05-30T01:15:28.731Z
648ac3d53470b17ccc90deaf
/avatars/35b678baa7a91959f40f704c5de2a3e1.svg
Ziteng Gao
user
sebgao
683906adf85de1fc563957d9
Mike Zheng Shou
extracted_pending
2025-05-30T01:15:28.731Z
63a55320ce5763e06f78519c
https://cdn-avatars.huggingface.co/v1/production/uploads/1671779060549-noauth.jpeg
Mike Shou
user
mikeshou
D-AR: Diffusion via Autoregressive Models
This paper presents Diffusion via Autoregressive models (D-AR), a new
paradigm recasting the image diffusion process as a vanilla autoregressive
procedure in the standard next-token-prediction fashion. We start by designing
the tokenizer that converts images into sequences of discrete tokens, where
tokens in different positions can be decoded into different diffusion denoising
steps in the pixel space. Thanks to the diffusion properties, these tokens
naturally follow a coarse-to-fine order, which directly lends itself to
autoregressive modeling. Therefore, we apply standard next-token prediction on
these tokens, without modifying any underlying designs (either causal masks or
training/inference strategies), and such sequential autoregressive token
generation directly mirrors the diffusion procedure in image space. That is,
once the autoregressive model generates an increment of tokens, we can directly
decode these tokens into the corresponding diffusion denoising step in the
streaming manner. Our pipeline naturally reveals several intriguing properties,
for example, it supports consistent previews when generating only a subset of
tokens and enables zero-shot layout-controlled synthesis. On the standard
ImageNet benchmark, our method achieves 2.09 FID using a 775M Llama backbone
with 256 discrete tokens. We hope our work can inspire future research on
unified autoregressive architectures of visual synthesis, especially with large
language models. Code and models will be available at
https://github.com/showlab/D-AR
Diffusion via Autoregressive models (D-AR) recasts the image diffusion process as a standard autoregressive task, achieving high-quality image generation with consistent previews and layout control using a large language model backbone.
Diffusion via Autoregressive models autoregressive procedure next-token-prediction tokenizer discrete tokens diffusion denoising coarse-to-fine order autoregressive modeling autoregressive token generation FID Llama backbone