title: Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale
    Prediction
summary: We present Visual AutoRegressive modeling (VAR), a new generation paradigm
  that redefines the autoregressive learning on images as coarse-to-fine
  "next-scale prediction" or "next-resolution prediction", diverging from the
  standard raster-scan "next-token prediction". This simple, intuitive
  methodology allows autoregressive (AR) transformers to learn visual
  distributions fast and generalize well: VAR, for the first time, makes AR
  models surpass diffusion transformers in image generation. On ImageNet 256x256
  benchmark, VAR significantly improve AR baseline by improving Frechet inception
  distance (FID) from 18.65 to 1.80, inception score (IS) from 80.4 to 356.4,
  with around 20x faster inference speed. It is also empirically verified that
  VAR outperforms the Diffusion Transformer (DiT) in multiple dimensions
  including image quality, inference speed, data efficiency, and scalability.
  Scaling up VAR models exhibits clear power-law scaling laws similar to those
  observed in LLMs, with linear correlation coefficients near -0.998 as solid
  evidence. VAR further showcases zero-shot generalization ability in downstream
  tasks including image in-painting, out-painting, and editing. These results
  suggest VAR has initially emulated the two important properties of LLMs:
  Scaling Laws and zero-shot task generalization. We have released all models and
  codes to promote the exploration of AR/VAR models for visual generation and
  unified learning.
paper_authors._id: 660e129262d63ad000c149de
paper_authors.name: Keyu Tian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-04T07:55:43.864Z
paper_authors._id: 60e73ffd06ad9ae5bbcfc52c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1625767908838-noauth.jpeg
paper_authors.fullname: Keyu Tian
paper_authors.type: user
paper_authors.user: keyu-tian
paper_authors._id: 660e129262d63ad000c149df
paper_authors.name: Yi Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-05T10:05:39.069Z
paper_authors._id: 6344dcb1cd37e44d9ed46508
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6344dcb1cd37e44d9ed46508/J92UKSxKR3iziD2WJfih4.jpeg
paper_authors.fullname: Yi Jiang
paper_authors.type: user
paper_authors.user: JiangYi
paper_authors._id: 660e129262d63ad000c149e0
paper_authors.name: Zehuan Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-25T07:38:11.619Z
paper_authors._id: 661a80af3557013b638061d5
paper_authors.avatarUrl: /avatars/4c551aeb223e257a5fc45b5b6c7ded49.svg
paper_authors.fullname: Zehuan Yuan
paper_authors.type: user
paper_authors.user: sweetrabor
paper_authors._id: 660e129262d63ad000c149e1
paper_authors.name: Bingyue Peng
paper_authors._id: 660e129262d63ad000c149e2
paper_authors.name: Liwei Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-25T07:38:33.015Z
paper_authors._id: 6497272b6401266eb6752b89
paper_authors.avatarUrl: /avatars/d94a011660f4cad5344756027840dc30.svg
paper_authors.fullname: Liwei Wang
paper_authors.type: user
paper_authors.user: asdfg80
paper_title: Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale
    Prediction
paper_summary: We present Visual AutoRegressive modeling (VAR), a new generation paradigm
  that redefines the autoregressive learning on images as coarse-to-fine
  "next-scale prediction" or "next-resolution prediction", diverging from the
  standard raster-scan "next-token prediction". This simple, intuitive
  methodology allows autoregressive (AR) transformers to learn visual
  distributions fast and generalize well: VAR, for the first time, makes AR
  models surpass diffusion transformers in image generation. On ImageNet 256x256
  benchmark, VAR significantly improve AR baseline by improving Frechet inception
  distance (FID) from 18.65 to 1.80, inception score (IS) from 80.4 to 356.4,
  with around 20x faster inference speed. It is also empirically verified that
  VAR outperforms the Diffusion Transformer (DiT) in multiple dimensions
  including image quality, inference speed, data efficiency, and scalability.
  Scaling up VAR models exhibits clear power-law scaling laws similar to those
  observed in LLMs, with linear correlation coefficients near -0.998 as solid
  evidence. VAR further showcases zero-shot generalization ability in downstream
  tasks including image in-painting, out-painting, and editing. These results
  suggest VAR has initially emulated the two important properties of LLMs:
  Scaling Laws and zero-shot task generalization. We have released all models and
  codes to promote the exploration of AR/VAR models for visual generation and
  unified learning.
paper_ai_summary: Visual AutoRegressive (VAR) modeling improves image generation performance and speed over autoregressive and diffusion transformer models, achieving superior results in image quality, data efficiency, and scalability.
paper_ai_keywords: Visual AutoRegressive modeling autoregressive learning next-scale prediction next-resolution prediction autoregressive transformers diffusion transformers Frechet inception distance inception score image quality inference speed data efficiency scalability LLMs scaling laws zero-shot generalization image in-painting out-painting image editing