Scalable Autoregressive Image Generation with Mamba
We introduce AiM, an autoregressive (AR) image generative model based on
Mamba architecture. AiM employs Mamba, a novel state-space model characterized
by its exceptional performance for long-sequence modeling with linear time
complexity, to supplant the commonly utilized Transformers in AR image
generation models, aiming to achieve both superior generation quality and
enhanced inference speed. Unlike existing methods that adapt Mamba to handle
two-dimensional signals via multi-directional scan, AiM directly utilizes the
next-token prediction paradigm for autoregressive image generation. This
approach circumvents the need for extensive modifications to enable Mamba to
learn 2D spatial representations. By implementing straightforward yet
strategically targeted modifications for visual generative tasks, we preserve
Mamba's core structure, fully exploiting its efficient long-sequence modeling
capabilities and scalability. We provide AiM models in various scales, with
parameter counts ranging from 148M to 1.3B. On the ImageNet1K 256*256
benchmark, our best AiM model achieves a FID of 2.21, surpassing all existing
AR models of comparable parameter counts and demonstrating significant
competitiveness against diffusion models, with 2 to 10 times faster inference
speed. Code is available at https://github.com/hp-l33/AiM
66c7f717c5c61e4baa81307e
Haopeng Li
claimed_verified
2024-08-26T07:15:40.855Z
64e86fbd0c2413c3571ef7a6
/avatars/960e17ba6a0d03fbd4700cd198adf5af.svg
Haopeng Li
user
hp-l33
66c7f717c5c61e4baa81307f
Jinyue Yang
66c7f717c5c61e4baa813080
Kexin Wang
66c7f717c5c61e4baa813081
Xuerui Qiu
claimed_verified
2025-02-18T09:37:08.473Z
66d98c2bc9758ee602cb2e22
/avatars/cea03f263d5ed292c4e558871112c61c.svg
Qiu
user
Xuerui123
66c7f717c5c61e4baa813082
Yuhong Chou
66c7f717c5c61e4baa813083
Xin Li
66c7f717c5c61e4baa813084
Guoqi Li
Scalable Autoregressive Image Generation with Mamba
We introduce AiM, an autoregressive (AR) image generative model based on
Mamba architecture. AiM employs Mamba, a novel state-space model characterized
by its exceptional performance for long-sequence modeling with linear time
complexity, to supplant the commonly utilized Transformers in AR image
generation models, aiming to achieve both superior generation quality and
enhanced inference speed. Unlike existing methods that adapt Mamba to handle
two-dimensional signals via multi-directional scan, AiM directly utilizes the
next-token prediction paradigm for autoregressive image generation. This
approach circumvents the need for extensive modifications to enable Mamba to
learn 2D spatial representations. By implementing straightforward yet
strategically targeted modifications for visual generative tasks, we preserve
Mamba's core structure, fully exploiting its efficient long-sequence modeling
capabilities and scalability. We provide AiM models in various scales, with
parameter counts ranging from 148M to 1.3B. On the ImageNet1K 256*256
benchmark, our best AiM model achieves a FID of 2.21, surpassing all existing
AR models of comparable parameter counts and demonstrating significant
competitiveness against diffusion models, with 2 to 10 times faster inference
speed. Code is available at https://github.com/hp-l33/AiM
AiM, an autoregressive image generative model using the Mamba state-space architecture, achieves superior quality and faster inference speed compared to existing AR and diffusion models.
autoregressive (AR) image generative model Mamba architecture state-space model long-sequence modeling linear time complexity Transformers next-token prediction paradigm visual generative tasks efficient long-sequence modeling scalability FID ImageNet1K diffusion models