title: Beyond Next-Token: Next-X Prediction for Autoregressive Visual
    Generation
summary: Autoregressive (AR) modeling, known for its next-token prediction paradigm,
  underpins state-of-the-art language and visual generative models.
  Traditionally, a ``token'' is treated as the smallest prediction unit, often a
  discrete symbol in language or a quantized patch in vision. However, the
  optimal token definition for 2D image structures remains an open question.
  Moreover, AR models suffer from exposure bias, where teacher forcing during
  training leads to error accumulation at inference. In this paper, we propose
  xAR, a generalized AR framework that extends the notion of a token to an entity
  X, which can represent an individual patch token, a cell (a ktimes k
  grouping of neighboring patches), a subsample (a non-local grouping of distant
  patches), a scale (coarse-to-fine resolution), or even a whole image.
  Additionally, we reformulate discrete token classification as
  continuous entity regression, leveraging flow-matching methods at each
  AR step. This approach conditions training on noisy entities instead of ground
  truth tokens, leading to Noisy Context Learning, which effectively alleviates
  exposure bias. As a result, xAR offers two key advantages: (1) it enables
  flexible prediction units that capture different contextual granularity and
  spatial structures, and (2) it mitigates exposure bias by avoiding reliance on
  teacher forcing. On ImageNet-256 generation benchmark, our base model, xAR-B
  (172M), outperforms DiT-XL/SiT-XL (675M) while achieving 20times faster
  inference. Meanwhile, xAR-H sets a new state-of-the-art with an FID of 1.24,
  running 2.2times faster than the previous best-performing model without
  relying on vision foundation modules (\eg, DINOv2) or advanced guidance
  interval sampling.
paper_authors._id: 67c1643aa4ccbde471532ba6
paper_authors.name: Sucheng Ren
paper_authors._id: 67c1643aa4ccbde471532ba7
paper_authors.name: Qihang Yu
paper_authors._id: 67c1643aa4ccbde471532ba8
paper_authors.name: Ju He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-17T08:45:43.724Z
paper_authors._id: 661c9059bcd78151e5c06ea1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/661c9059bcd78151e5c06ea1/27bfNo1LZeZQ77vWuAa10.png
paper_authors.fullname: Ju He
paper_authors.type: user
paper_authors.user: turkeyju
paper_authors._id: 67c1643aa4ccbde471532ba9
paper_authors.name: Xiaohui Shen
paper_authors._id: 67c1643aa4ccbde471532baa
paper_authors.name: Alan Yuille
paper_authors._id: 67c1643aa4ccbde471532bab
paper_authors.name: Liang-Chieh Chen
paper_title: Beyond Next-Token: Next-X Prediction for Autoregressive Visual
    Generation
paper_summary: Autoregressive (AR) modeling, known for its next-token prediction paradigm,
  underpins state-of-the-art language and visual generative models.
  Traditionally, a ``token'' is treated as the smallest prediction unit, often a
  discrete symbol in language or a quantized patch in vision. However, the
  optimal token definition for 2D image structures remains an open question.
  Moreover, AR models suffer from exposure bias, where teacher forcing during
  training leads to error accumulation at inference. In this paper, we propose
  xAR, a generalized AR framework that extends the notion of a token to an entity
  X, which can represent an individual patch token, a cell (a ktimes k
  grouping of neighboring patches), a subsample (a non-local grouping of distant
  patches), a scale (coarse-to-fine resolution), or even a whole image.
  Additionally, we reformulate discrete token classification as
  continuous entity regression, leveraging flow-matching methods at each
  AR step. This approach conditions training on noisy entities instead of ground
  truth tokens, leading to Noisy Context Learning, which effectively alleviates
  exposure bias. As a result, xAR offers two key advantages: (1) it enables
  flexible prediction units that capture different contextual granularity and
  spatial structures, and (2) it mitigates exposure bias by avoiding reliance on
  teacher forcing. On ImageNet-256 generation benchmark, our base model, xAR-B
  (172M), outperforms DiT-XL/SiT-XL (675M) while achieving 20times faster
  inference. Meanwhile, xAR-H sets a new state-of-the-art with an FID of 1.24,
  running 2.2times faster than the previous best-performing model without
  relying on vision foundation modules (\eg, DINOv2) or advanced guidance
  interval sampling.
paper_ai_keywords: autoregressive (AR) modeling next-token prediction language and visual generative models token discrete symbol quantized patch exposure bias teacher forcing xAR generalized AR framework entity (X) patch token cell contiguous grouping of neighboring patches subsample non-local grouping of distant patches scale coarse-to-fine resolution whole image continuous entity regression flow-matching methods Noisy Context Learning ImageNet-256 benchmark FID (Fréchet Inception Distance) DiT-XL/SiT-XL vision foundation modules (e.g., DINOv2) advanced guidance interval sampling