title: Neighboring Autoregressive Modeling for Efficient Visual Generation
summary: Visual autoregressive models typically adhere to a raster-order ``next-token
  prediction" paradigm, which overlooks the spatial and temporal locality
  inherent in visual content. Specifically, visual tokens exhibit significantly
  stronger correlations with their spatially or temporally adjacent tokens
  compared to those that are distant. In this paper, we propose Neighboring
  Autoregressive Modeling (NAR), a novel paradigm that formulates autoregressive
  visual generation as a progressive outpainting procedure, following a
  near-to-far ``next-neighbor prediction" mechanism. Starting from an initial
  token, the remaining tokens are decoded in ascending order of their Manhattan
  distance from the initial token in the spatial-temporal space, progressively
  expanding the boundary of the decoded region. To enable parallel prediction of
  multiple adjacent tokens in the spatial-temporal space, we introduce a set of
  dimension-oriented decoding heads, each predicting the next token along a
  mutually orthogonal dimension. During inference, all tokens adjacent to the
  decoded tokens are processed in parallel, substantially reducing the model
  forward steps for generation. Experiments on ImageNet256times 256 and UCF101
  demonstrate that NAR achieves 2.4times and 8.6times higher throughput
  respectively, while obtaining superior FID/FVD scores for both image and video
  generation tasks compared to the PAR-4X approach. When evaluating on
  text-to-image generation benchmark GenEval, NAR with 0.8B parameters
  outperforms Chameleon-7B while using merely 0.4 of the training data. Code is
  available at https://github.com/ThisisBillhe/NAR.
paper_authors._id: 67d7e9bd93b8599318993db2
paper_authors.name: Yefei He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T14:54:40.739Z
paper_authors._id: 65a88c3d26598b995531fff1
paper_authors.avatarUrl: /avatars/b1a524857d8572d0405476661b434160.svg
paper_authors.fullname: Yefei He
paper_authors.type: user
paper_authors.user: yefly
paper_authors._id: 67d7e9bd93b8599318993db3
paper_authors.name: Yuanyu He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T14:55:03.703Z
paper_authors._id: 668a94d30b04331a0b6dc457
paper_authors.avatarUrl: /avatars/66a311a7474e9d297231868dbbd6a18c.svg
paper_authors.fullname: heyuanyu
paper_authors.type: user
paper_authors.user: heyuanyu
paper_authors._id: 67d7e9bd93b8599318993db4
paper_authors.name: Shaoxuan He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T14:54:33.115Z
paper_authors._id: 66365ad3ea6f5788b32a2fb4
paper_authors.avatarUrl: /avatars/99a7aebc1d022b52d7692494bdd2a1c7.svg
paper_authors.fullname: Shaoxuan He
paper_authors.type: user
paper_authors.user: shxhe
paper_authors._id: 67d7e9bd93b8599318993db5
paper_authors.name: Feng Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-17T14:42:15.701Z
paper_authors._id: 64574380f3ef144c0e69d484
paper_authors.avatarUrl: /avatars/a0a84757cb0bf09c24291803e1389b49.svg
paper_authors.fullname: Feng Chen
paper_authors.type: user
paper_authors.user: chenfeng1271
paper_authors._id: 67d7e9bd93b8599318993db6
paper_authors.name: Hong Zhou
paper_authors._id: 67d7e9bd93b8599318993db7
paper_authors.name: Kaipeng Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:52:25.151Z
paper_authors._id: 65f1713552c38a91e0a445e8
paper_authors.avatarUrl: /avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
paper_authors.fullname: kaipeng
paper_authors.type: user
paper_authors.user: kpzhang996
paper_authors._id: 67d7e9bd93b8599318993db8
paper_authors.name: Bohan Zhuang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T14:55:23.366Z
paper_authors._id: 65a8a828ef14f9e6037dfad2
paper_authors.avatarUrl: /avatars/55fdc0393b2e37ac998ddbdb1e1ff636.svg
paper_authors.fullname: Bohan Zhuang
paper_authors.type: user
paper_authors.user: BohanZ
paper_title: Neighboring Autoregressive Modeling for Efficient Visual Generation
paper_summary: Visual autoregressive models typically adhere to a raster-order ``next-token
  prediction" paradigm, which overlooks the spatial and temporal locality
  inherent in visual content. Specifically, visual tokens exhibit significantly
  stronger correlations with their spatially or temporally adjacent tokens
  compared to those that are distant. In this paper, we propose Neighboring
  Autoregressive Modeling (NAR), a novel paradigm that formulates autoregressive
  visual generation as a progressive outpainting procedure, following a
  near-to-far ``next-neighbor prediction" mechanism. Starting from an initial
  token, the remaining tokens are decoded in ascending order of their Manhattan
  distance from the initial token in the spatial-temporal space, progressively
  expanding the boundary of the decoded region. To enable parallel prediction of
  multiple adjacent tokens in the spatial-temporal space, we introduce a set of
  dimension-oriented decoding heads, each predicting the next token along a
  mutually orthogonal dimension. During inference, all tokens adjacent to the
  decoded tokens are processed in parallel, substantially reducing the model
  forward steps for generation. Experiments on ImageNet256times 256 and UCF101
  demonstrate that NAR achieves 2.4times and 8.6times higher throughput
  respectively, while obtaining superior FID/FVD scores for both image and video
  generation tasks compared to the PAR-4X approach. When evaluating on
  text-to-image generation benchmark GenEval, NAR with 0.8B parameters
  outperforms Chameleon-7B while using merely 0.4 of the training data. Code is
  available at https://github.com/ThisisBillhe/NAR.
paper_ai_summary: Neighboring Autoregressive Modeling (NAR) improves visual generation throughput and quality by predicting tokens in a near-to-far spatial-temporal order and using dimension-oriented decoding heads.
paper_ai_keywords: visual autoregressive models next-token prediction spatial locality temporal locality visual tokens Neighboring Autoregressive Modeling NAR progressive outpainting Manhattan distance spatial-temporal space dimension-oriented decoding heads FID scores FVD scores text-to-image generation GenEval Chameleon-7B