title: GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for
    Autoregressive Image Generation
summary: In autoregressive (AR) image generation, visual tokenizers compress images
  into compact discrete latent tokens, enabling efficient training of downstream
  autoregressive models for visual generation via next-token prediction. While
  scaling visual tokenizers improves image reconstruction quality, it often
  degrades downstream generation quality -- a challenge not adequately addressed
  in existing literature. To address this, we introduce GigaTok, the first
  approach to simultaneously improve image reconstruction, generation, and
  representation learning when scaling visual tokenizers. We identify the growing
  complexity of latent space as the key factor behind the reconstruction vs.
  generation dilemma. To mitigate this, we propose semantic regularization, which
  aligns tokenizer features with semantically consistent features from a
  pre-trained visual encoder. This constraint prevents excessive latent space
  complexity during scaling, yielding consistent improvements in both
  reconstruction and downstream autoregressive generation. Building on semantic
  regularization, we explore three key practices for scaling tokenizers:(1) using
  1D tokenizers for better scalability, (2) prioritizing decoder scaling when
  expanding both encoder and decoder, and (3) employing entropy loss to stabilize
  training for billion-scale tokenizers. By scaling to 3 space billion
  parameters, GigaTok achieves state-of-the-art performance in reconstruction,
  downstream AR generation, and downstream AR representation quality.
paper_authors._id: 67fc8e37864dfcbd93d3b802
paper_authors.name: Tianwei Xiong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-14T09:46:29.330Z
paper_authors._id: 668125557b50b433cda2a211
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/668125557b50b433cda2a211/j3z3wT5Rv9IyUKtbzQpnc.png
paper_authors.fullname: Tianwei Xiong
paper_authors.type: user
paper_authors.user: YuuTennYi
paper_authors._id: 67fc8e37864dfcbd93d3b803
paper_authors.name: Jun Hao Liew
paper_authors._id: 67fc8e37864dfcbd93d3b804
paper_authors.name: Zilong Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-14T07:36:55.605Z
paper_authors._id: 63a430ec84a6a25c65bff20c
paper_authors.avatarUrl: /avatars/3ab51214c491f28d8fe6b70c373d72aa.svg
paper_authors.fullname: Zilong Huang
paper_authors.type: user
paper_authors.user: speedinghzl
paper_authors._id: 67fc8e37864dfcbd93d3b805
paper_authors.name: Jiashi Feng
paper_authors._id: 67fc8e37864dfcbd93d3b806
paper_authors.name: Xihui Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-14T09:46:26.140Z
paper_authors._id: 65d5ec74cd05bc1eaa125040
paper_authors.avatarUrl: /avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
paper_authors.fullname: Xihui Liu
paper_authors.type: user
paper_authors.user: XihuiLiu
paper_title: GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for
    Autoregressive Image Generation
paper_summary: In autoregressive (AR) image generation, visual tokenizers compress images
  into compact discrete latent tokens, enabling efficient training of downstream
  autoregressive models for visual generation via next-token prediction. While
  scaling visual tokenizers improves image reconstruction quality, it often
  degrades downstream generation quality -- a challenge not adequately addressed
  in existing literature. To address this, we introduce GigaTok, the first
  approach to simultaneously improve image reconstruction, generation, and
  representation learning when scaling visual tokenizers. We identify the growing
  complexity of latent space as the key factor behind the reconstruction vs.
  generation dilemma. To mitigate this, we propose semantic regularization, which
  aligns tokenizer features with semantically consistent features from a
  pre-trained visual encoder. This constraint prevents excessive latent space
  complexity during scaling, yielding consistent improvements in both
  reconstruction and downstream autoregressive generation. Building on semantic
  regularization, we explore three key practices for scaling tokenizers:(1) using
  1D tokenizers for better scalability, (2) prioritizing decoder scaling when
  expanding both encoder and decoder, and (3) employing entropy loss to stabilize
  training for billion-scale tokenizers. By scaling to 3 space billion
  parameters, GigaTok achieves state-of-the-art performance in reconstruction,
  downstream AR generation, and downstream AR representation quality.
paper_ai_summary: GigaTok improves image reconstruction, generation, and representation quality by scaling visual tokenizers with semantic regularization and specific architectural practices.
paper_ai_keywords: autoregressive (AR) image generation visual tokenizers latent tokens image reconstruction downstream generation latent space semantic regularization pre-trained visual encoder 1D tokenizers decoder scaling entropy loss billion-scale tokenizers