Autoregressive Image Generation with Randomized Parallel Decoding
We introduce ARPG, a novel visual autoregressive model that enables
randomized parallel generation, addressing the inherent limitations of
conventional raster-order approaches, which hinder inference efficiency and
zero-shot generalization due to their sequential, predefined token generation
order. Our key insight is that effective random-order modeling necessitates
explicit guidance for determining the position of the next predicted token. To
this end, we propose a novel guided decoding framework that decouples
positional guidance from content representation, encoding them separately as
queries and key-value pairs. By directly incorporating this guidance into the
causal attention mechanism, our approach enables fully random-order training
and generation, eliminating the need for bidirectional attention. Consequently,
ARPG readily generalizes to zero-shot tasks such as image inpainting,
outpainting, and resolution expansion. Furthermore, it supports parallel
inference by concurrently processing multiple queries using a shared KV cache.
On the ImageNet-1K 256 benchmark, our approach attains an FID of 1.94 with only
64 sampling steps, achieving over a 20-fold increase in throughput while
reducing memory consumption by over 75% compared to representative recent
autoregressive models at a similar scale.
67d3a952687a7a8a4963a030
Haopeng Li
claimed_verified
2025-03-14T08:56:10.236Z
64e86fbd0c2413c3571ef7a6
/avatars/960e17ba6a0d03fbd4700cd198adf5af.svg
Haopeng Li
user
hp-l33
67d3a952687a7a8a4963a031
Jinyue Yang
67d3a952687a7a8a4963a032
Guoqi Li
67d3a952687a7a8a4963a033
Huan Wang
claimed_verified
2025-03-17T14:42:36.608Z
62b624f3b52bef716e248fd7
https://cdn-avatars.huggingface.co/v1/production/uploads/62b624f3b52bef716e248fd7/AllcccKH-eBWduA8KVnOQ.png
Huan Wang
user
Huan-WhoRegisteredMyName
Autoregressive Image Generation with Randomized Parallel Decoding
We introduce ARPG, a novel visual autoregressive model that enables
randomized parallel generation, addressing the inherent limitations of
conventional raster-order approaches, which hinder inference efficiency and
zero-shot generalization due to their sequential, predefined token generation
order. Our key insight is that effective random-order modeling necessitates
explicit guidance for determining the position of the next predicted token. To
this end, we propose a novel guided decoding framework that decouples
positional guidance from content representation, encoding them separately as
queries and key-value pairs. By directly incorporating this guidance into the
causal attention mechanism, our approach enables fully random-order training
and generation, eliminating the need for bidirectional attention. Consequently,
ARPG readily generalizes to zero-shot tasks such as image inpainting,
outpainting, and resolution expansion. Furthermore, it supports parallel
inference by concurrently processing multiple queries using a shared KV cache.
On the ImageNet-1K 256 benchmark, our approach attains an FID of 1.94 with only
64 sampling steps, achieving over a 20-fold increase in throughput while
reducing memory consumption by over 75% compared to representative recent
autoregressive models at a similar scale.
ARPG, a visual autoregressive model with randomized parallel generation, improves inference efficiency and zero-shot generalization by decoupling positional guidance and content representation into queries and key-value pairs within a causal attention mechanism.
visual autoregressive model randomized parallel generation raster-order approaches zero-shot generalization guided decoding framework causal attention mechanism ImageNet-1K 256 benchmark FID sampling steps throughput memory consumption