title: Autoregressive Model Beats Diffusion: Llama for Scalable Image
    Generation
summary: We introduce LlamaGen, a new family of image generation models that apply
  original ``next-token prediction'' paradigm of large language models to visual
  generation domain. It is an affirmative answer to whether vanilla
  autoregressive models, e.g., Llama, without inductive biases on visual signals
  can achieve state-of-the-art image generation performance if scaling properly.
  We reexamine design spaces of image tokenizers, scalability properties of image
  generation models, and their training data quality. The outcome of this
  exploration consists of: (1) An image tokenizer with downsample ratio of 16,
  reconstruction quality of 0.94 rFID and codebook usage of 97% on ImageNet
  benchmark. (2) A series of class-conditional image generation models ranging
  from 111M to 3.1B parameters, achieving 2.18 FID on ImageNet 256x256
  benchmarks, outperforming the popular diffusion models such as LDM, DiT. (3) A
  text-conditional image generation model with 775M parameters, from two-stage
  training on LAION-COCO and high aesthetics quality images, demonstrating
  competitive performance of visual quality and text alignment. (4) We verify the
  effectiveness of LLM serving frameworks in optimizing the inference speed of
  image generation models and achieve 326% - 414% speedup. We release all models
  and codes to facilitate open-source community of visual generation and
  multimodal foundation models.
paper_authors._id: 6667c71be490a870be48da80
paper_authors.name: Peize Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:36:13.868Z
paper_authors._id: 640dc9bf8512ec51d7f0ac1a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640dc9bf8512ec51d7f0ac1a/sT4rdEoQbzfW6D3xDVdqt.jpeg
paper_authors.fullname: peizesun
paper_authors.type: user
paper_authors.user: peizesun
paper_authors._id: 6667c71be490a870be48da81
paper_authors.name: Yi Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:39:40.191Z
paper_authors._id: 6344dcb1cd37e44d9ed46508
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6344dcb1cd37e44d9ed46508/J92UKSxKR3iziD2WJfih4.jpeg
paper_authors.fullname: Yi Jiang
paper_authors.type: user
paper_authors.user: JiangYi
paper_authors._id: 6667c71be490a870be48da82
paper_authors.name: Shoufa Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-11T06:59:01.343Z
paper_authors._id: 6412a33900634c4fe9873652
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6412a33900634c4fe9873652/Nmn_yRA1gGD2VO1YbSOYF.jpeg
paper_authors.fullname: Shoufa Chen
paper_authors.type: user
paper_authors.user: ShoufaChen
paper_authors._id: 6667c71be490a870be48da83
paper_authors.name: Shilong Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:40:05.731Z
paper_authors._id: 6424ffce46d202ad3d918a67
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6424ffce46d202ad3d918a67/gmYmOA072fP_5cJLc9Qs4.jpeg
paper_authors.fullname: Shilong Zhang
paper_authors.type: user
paper_authors.user: shilongz
paper_authors._id: 6667c71be490a870be48da84
paper_authors.name: Bingyue Peng
paper_authors._id: 6667c71be490a870be48da85
paper_authors.name: Ping Luo
paper_authors._id: 6667c71be490a870be48da86
paper_authors.name: Zehuan Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:40:56.271Z
paper_authors._id: 661a80af3557013b638061d5
paper_authors.avatarUrl: /avatars/4c551aeb223e257a5fc45b5b6c7ded49.svg
paper_authors.fullname: Zehuan Yuan
paper_authors.type: user
paper_authors.user: sweetrabor
paper_title: Autoregressive Model Beats Diffusion: Llama for Scalable Image
    Generation
paper_summary: We introduce LlamaGen, a new family of image generation models that apply
  original ``next-token prediction'' paradigm of large language models to visual
  generation domain. It is an affirmative answer to whether vanilla
  autoregressive models, e.g., Llama, without inductive biases on visual signals
  can achieve state-of-the-art image generation performance if scaling properly.
  We reexamine design spaces of image tokenizers, scalability properties of image
  generation models, and their training data quality. The outcome of this
  exploration consists of: (1) An image tokenizer with downsample ratio of 16,
  reconstruction quality of 0.94 rFID and codebook usage of 97% on ImageNet
  benchmark. (2) A series of class-conditional image generation models ranging
  from 111M to 3.1B parameters, achieving 2.18 FID on ImageNet 256x256
  benchmarks, outperforming the popular diffusion models such as LDM, DiT. (3) A
  text-conditional image generation model with 775M parameters, from two-stage
  training on LAION-COCO and high aesthetics quality images, demonstrating
  competitive performance of visual quality and text alignment. (4) We verify the
  effectiveness of LLM serving frameworks in optimizing the inference speed of
  image generation models and achieve 326% - 414% speedup. We release all models
  and codes to facilitate open-source community of visual generation and
  multimodal foundation models.
paper_ai_keywords: next-token prediction Llama autoregressive models inductive biases image tokenizers scalability training data quality downsample ratio rFID codebook usage ImageNet benchmark class-conditional image generation text-conditional image generation LAION-COCO visual quality text alignment image continuation models LLM serving frameworks inference speedup