title: MaskBit: Embedding-free Image Generation via Bit Tokens
summary: Masked transformer models for class-conditional image generation have become
  a compelling alternative to diffusion models. Typically comprising two stages -
  an initial VQGAN model for transitioning between latent space and image space,
  and a subsequent Transformer model for image generation within latent space -
  these frameworks offer promising avenues for image synthesis. In this study, we
  present two primary contributions: Firstly, an empirical and systematic
  examination of VQGANs, leading to a modernized VQGAN. Secondly, a novel
  embedding-free generation network operating directly on bit tokens - a binary
  quantized representation of tokens with rich semantics. The first contribution
  furnishes a transparent, reproducible, and high-performing VQGAN model,
  enhancing accessibility and matching the performance of current
  state-of-the-art methods while revealing previously undisclosed details. The
  second contribution demonstrates that embedding-free image generation using bit
  tokens achieves a new state-of-the-art FID of 1.52 on the ImageNet 256x256
  benchmark, with a compact generator model of mere 305M parameters.
paper_authors._id: 66f38fc183174918394174bd
paper_authors.name: Mark Weber
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-25T18:24:20.402Z
paper_authors._id: 651b3b452930812657c154ce
paper_authors.avatarUrl: /avatars/d424ef67b20e4277c4af44e05d1ea188.svg
paper_authors.fullname: Mark Weber
paper_authors.type: user
paper_authors.user: markweber
paper_authors._id: 66f38fc183174918394174be
paper_authors.name: Lijun Yu
paper_authors._id: 66f38fc183174918394174bf
paper_authors.name: Qihang Yu
paper_authors._id: 66f38fc183174918394174c0
paper_authors.name: Xueqing Deng
paper_authors._id: 66f38fc183174918394174c1
paper_authors.name: Xiaohui Shen
paper_authors._id: 66f38fc183174918394174c2
paper_authors.name: Daniel Cremers
paper_authors._id: 66f38fc183174918394174c3
paper_authors.name: Liang-Chieh Chen
paper_title: MaskBit: Embedding-free Image Generation via Bit Tokens
paper_summary: Masked transformer models for class-conditional image generation have become
  a compelling alternative to diffusion models. Typically comprising two stages -
  an initial VQGAN model for transitioning between latent space and image space,
  and a subsequent Transformer model for image generation within latent space -
  these frameworks offer promising avenues for image synthesis. In this study, we
  present two primary contributions: Firstly, an empirical and systematic
  examination of VQGANs, leading to a modernized VQGAN. Secondly, a novel
  embedding-free generation network operating directly on bit tokens - a binary
  quantized representation of tokens with rich semantics. The first contribution
  furnishes a transparent, reproducible, and high-performing VQGAN model,
  enhancing accessibility and matching the performance of current
  state-of-the-art methods while revealing previously undisclosed details. The
  second contribution demonstrates that embedding-free image generation using bit
  tokens achieves a new state-of-the-art FID of 1.52 on the ImageNet 256x256
  benchmark, with a compact generator model of mere 305M parameters.
paper_ai_summary: Modernized VQGAN and embedding-free bit token based Transformer models achieve state-of-the-art results in class-conditional image generation.
paper_ai_keywords: masked transformer models VQGAN latent space image space embedding-free generation bit tokens FID ImageNet 256x256