Parallelized Autoregressive Visual Generation
Autoregressive models have emerged as a powerful approach for visual
generation but suffer from slow inference speed due to their sequential
token-by-token prediction process. In this paper, we propose a simple yet
effective approach for parallelized autoregressive visual generation that
improves generation efficiency while preserving the advantages of
autoregressive modeling. Our key insight is that parallel generation depends on
visual token dependencies-tokens with weak dependencies can be generated in
parallel, while strongly dependent adjacent tokens are difficult to generate
together, as their independent sampling may lead to inconsistencies. Based on
this observation, we develop a parallel generation strategy that generates
distant tokens with weak dependencies in parallel while maintaining sequential
generation for strongly dependent local tokens. Our approach can be seamlessly
integrated into standard autoregressive models without modifying the
architecture or tokenizer. Experiments on ImageNet and UCF-101 demonstrate that
our method achieves a 3.6x speedup with comparable quality and up to 9.5x
speedup with minimal quality degradation across both image and video generation
tasks. We hope this work will inspire future research in efficient visual
generation and unified autoregressive modeling. Project page:
https://epiphqny.github.io/PAR-project.
6764fd9b10330426aecdded7
Yuqing Wang
claimed_verified
2024-12-23T11:11:21.169Z
63ea23b9dedfeebe54d02bdf
/avatars/4d9f9a546aa8c63e277161ea700075c4.svg
Yuqing Wang
user
Epiphqny
6764fd9b10330426aecdded8
Shuhuai Ren
claimed_verified
2024-12-23T11:11:23.352Z
60d2e681b8448e1785bbda06
https://cdn-avatars.huggingface.co/v1/production/uploads/1624434302056-noauth.jpeg
Shuhuai Ren
user
ShuhuaiRen
6764fd9b10330426aecdded9
Zhijie Lin
admin_assigned
2024-12-23T11:30:34.250Z
64415957bd0c9726529802f6
/avatars/1132d1ee68fb58ec635d57c8175caacd.svg
Zhijie Lin
user
Ikuinen
6764fd9b10330426aecddeda
Yujin Han
6764fd9b10330426aecddedb
Haoyuan Guo
6764fd9b10330426aecddedc
Zhenheng Yang
admin_assigned
2024-12-23T11:24:14.038Z
6421183b69a2c2933882d652
/avatars/66813a8fa22915087cccd4dbfb945ca7.svg
Zhenheng Yang
user
zhenheny
6764fd9b10330426aecddedd
Difan Zou
6764fd9b10330426aecddede
Jiashi Feng
admin_assigned
2024-12-23T11:23:58.910Z
67298e44017b96a1d0101dc4
/avatars/1f8ed1a3e911e6a3021087b9371d284c.svg
Jiashi Feng
user
jshfeng
6764fd9b10330426aecddedf
Xihui Liu
claimed_verified
2024-12-23T11:11:19.211Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
Parallelized Autoregressive Visual Generation
Autoregressive models have emerged as a powerful approach for visual
generation but suffer from slow inference speed due to their sequential
token-by-token prediction process. In this paper, we propose a simple yet
effective approach for parallelized autoregressive visual generation that
improves generation efficiency while preserving the advantages of
autoregressive modeling. Our key insight is that parallel generation depends on
visual token dependencies-tokens with weak dependencies can be generated in
parallel, while strongly dependent adjacent tokens are difficult to generate
together, as their independent sampling may lead to inconsistencies. Based on
this observation, we develop a parallel generation strategy that generates
distant tokens with weak dependencies in parallel while maintaining sequential
generation for strongly dependent local tokens. Our approach can be seamlessly
integrated into standard autoregressive models without modifying the
architecture or tokenizer. Experiments on ImageNet and UCF-101 demonstrate that
our method achieves a 3.6x speedup with comparable quality and up to 9.5x
speedup with minimal quality degradation across both image and video generation
tasks. We hope this work will inspire future research in efficient visual
generation and unified autoregressive modeling. Project page:
https://epiphqny.github.io/PAR-project.
A parallel generation strategy for autoregressive models improves inference speed without significantly compromising quality in visual generation tasks.
autoregressive models parallelized autoregressive visual generation visual token dependencies sequential generation distant tokens local tokens ImageNet UCF-101 parallel generation strategy efficient visual generation unified autoregressive modeling