Locality-aware Parallel Decoding for Efficient Autoregressive Image
  Generation
We present Locality-aware Parallel Decoding (LPD) to accelerate
autoregressive image generation. Traditional autoregressive image generation
relies on next-patch prediction, a memory-bound process that leads to high
latency. Existing works have tried to parallelize next-patch prediction by
shifting to multi-patch prediction to accelerate the process, but only achieved
limited parallelization. To achieve high parallelization while maintaining
generation quality, we introduce two key techniques: (1) Flexible Parallelized
Autoregressive Modeling, a novel architecture that enables arbitrary generation
ordering and degrees of parallelization. It uses learnable position query
tokens to guide generation at target positions while ensuring mutual visibility
among concurrently generated tokens for consistent parallel decoding. (2)
Locality-aware Generation Ordering, a novel schedule that forms groups to
minimize intra-group dependencies and maximize contextual support, enhancing
generation quality. With these designs, we reduce the generation steps from 256
to 20 (256times256 res.) and 1024 to 48 (512times512 res.) without
compromising quality on the ImageNet class-conditional generation, and
achieving at least 3.4times lower latency than previous parallelized
autoregressive models.
686633d28c83dab5f72d1f39
Zhuoyang Zhang
686633d28c83dab5f72d1f3a
Luke J. Huang
686633d28c83dab5f72d1f3b
Chengyue Wu
claimed_verified
2025-07-03T16:19:00.266Z
617526c9de8feb54b0ce45ad
/avatars/7faf8c6f71fc318a0113d780d376c381.svg
Wu Chengyue
user
WuChengyue
686633d28c83dab5f72d1f3c
Shang Yang
686633d28c83dab5f72d1f3d
Kelly Peng
686633d28c83dab5f72d1f3e
Yao Lu
686633d28c83dab5f72d1f3f
Song Han
Locality-aware Parallel Decoding for Efficient Autoregressive Image
  Generation
We present Locality-aware Parallel Decoding (LPD) to accelerate
autoregressive image generation. Traditional autoregressive image generation
relies on next-patch prediction, a memory-bound process that leads to high
latency. Existing works have tried to parallelize next-patch prediction by
shifting to multi-patch prediction to accelerate the process, but only achieved
limited parallelization. To achieve high parallelization while maintaining
generation quality, we introduce two key techniques: (1) Flexible Parallelized
Autoregressive Modeling, a novel architecture that enables arbitrary generation
ordering and degrees of parallelization. It uses learnable position query
tokens to guide generation at target positions while ensuring mutual visibility
among concurrently generated tokens for consistent parallel decoding. (2)
Locality-aware Generation Ordering, a novel schedule that forms groups to
minimize intra-group dependencies and maximize contextual support, enhancing
generation quality. With these designs, we reduce the generation steps from 256
to 20 (256times256 res.) and 1024 to 48 (512times512 res.) without
compromising quality on the ImageNet class-conditional generation, and
achieving at least 3.4times lower latency than previous parallelized
autoregressive models.
Locality-aware Parallel Decoding accelerates autoregressive image generation by enabling flexible parallelization and optimized generation ordering, significantly reducing latency without compromising quality.
Locality-aware Parallel Decoding Flexible Parallelized Autoregressive Modeling learnable position query tokens Locality-aware Generation Ordering autoregressive image generation next-patch prediction multi-patch prediction generation quality ImageNet class-conditional generation