MaskBit: Embedding-free Image Generation via Bit Tokens
Masked transformer models for class-conditional image generation have become
a compelling alternative to diffusion models. Typically comprising two stages -
an initial VQGAN model for transitioning between latent space and image space,
and a subsequent Transformer model for image generation within latent space -
these frameworks offer promising avenues for image synthesis. In this study, we
present two primary contributions: Firstly, an empirical and systematic
examination of VQGANs, leading to a modernized VQGAN. Secondly, a novel
embedding-free generation network operating directly on bit tokens - a binary
quantized representation of tokens with rich semantics. The first contribution
furnishes a transparent, reproducible, and high-performing VQGAN model,
enhancing accessibility and matching the performance of current
state-of-the-art methods while revealing previously undisclosed details. The
second contribution demonstrates that embedding-free image generation using bit
tokens achieves a new state-of-the-art FID of 1.52 on the ImageNet 256x256
benchmark, with a compact generator model of mere 305M parameters.
66f38fc183174918394174bd
Mark Weber
claimed_verified
2024-09-25T18:24:20.402Z
651b3b452930812657c154ce
/avatars/d424ef67b20e4277c4af44e05d1ea188.svg
Mark Weber
user
markweber
66f38fc183174918394174be
Lijun Yu
66f38fc183174918394174bf
Qihang Yu
66f38fc183174918394174c0
Xueqing Deng
66f38fc183174918394174c1
Xiaohui Shen
66f38fc183174918394174c2
Daniel Cremers
66f38fc183174918394174c3
Liang-Chieh Chen
MaskBit: Embedding-free Image Generation via Bit Tokens
Masked transformer models for class-conditional image generation have become
a compelling alternative to diffusion models. Typically comprising two stages -
an initial VQGAN model for transitioning between latent space and image space,
and a subsequent Transformer model for image generation within latent space -
these frameworks offer promising avenues for image synthesis. In this study, we
present two primary contributions: Firstly, an empirical and systematic
examination of VQGANs, leading to a modernized VQGAN. Secondly, a novel
embedding-free generation network operating directly on bit tokens - a binary
quantized representation of tokens with rich semantics. The first contribution
furnishes a transparent, reproducible, and high-performing VQGAN model,
enhancing accessibility and matching the performance of current
state-of-the-art methods while revealing previously undisclosed details. The
second contribution demonstrates that embedding-free image generation using bit
tokens achieves a new state-of-the-art FID of 1.52 on the ImageNet 256x256
benchmark, with a compact generator model of mere 305M parameters.
Modernized VQGAN and embedding-free bit token based Transformer models achieve state-of-the-art results in class-conditional image generation.
masked transformer models VQGAN latent space image space embedding-free generation bit tokens FID ImageNet 256x256