title: CODA: Repurposing Continuous VAEs for Discrete Tokenization
summary: Discrete visual tokenizers transform images into a sequence of tokens,
  enabling token-based visual generation akin to language models. However, this
  process is inherently challenging, as it requires both compressing visual
  signals into a compact representation and discretizing them into a fixed set of
  codes. Traditional discrete tokenizers typically learn the two tasks jointly,
  often leading to unstable training, low codebook utilization, and limited
  reconstruction quality. In this paper, we introduce
  CODA(COntinuous-to-Discrete Adaptation), a
  framework that decouples compression and discretization. Instead of training
  discrete tokenizers from scratch, CODA adapts off-the-shelf continuous VAEs --
  already optimized for perceptual compression -- into discrete tokenizers via a
  carefully designed discretization process. By primarily focusing on
  discretization, CODA ensures stable and efficient training while retaining the
  strong visual fidelity of continuous VAEs. Empirically, with 6
  times less training budget than standard VQGAN, our approach achieves a
  remarkable codebook utilization of 100% and notable reconstruction FID (rFID)
  of 0.43 and 1.34 for 8 times and 16 times
  compression on ImageNet 256times 256 benchmark.
paper_authors._id: 67e2b2c1a46a0939c33119b5
paper_authors.name: Zeyu Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-25T14:35:58.054Z
paper_authors._id: 64f2a14396a5ffb820ab39a8
paper_authors.avatarUrl: /avatars/3d7c3be17c53eb52bd8a75d3c51e6891.svg
paper_authors.fullname: Zeyu Liu
paper_authors.type: user
paper_authors.user: shuidi0020
paper_authors._id: 67e2b2c1a46a0939c33119b6
paper_authors.name: Zanlin Ni
paper_authors._id: 67e2b2c1a46a0939c33119b7
paper_authors.name: Yeguo Hua
paper_authors._id: 67e2b2c1a46a0939c33119b8
paper_authors.name: Xin Deng
paper_authors._id: 67e2b2c1a46a0939c33119b9
paper_authors.name: Xiao Ma
paper_authors._id: 67e2b2c1a46a0939c33119ba
paper_authors.name: Cheng Zhong
paper_authors._id: 67e2b2c1a46a0939c33119bb
paper_authors.name: Gao Huang
paper_title: CODA: Repurposing Continuous VAEs for Discrete Tokenization
paper_summary: Discrete visual tokenizers transform images into a sequence of tokens,
  enabling token-based visual generation akin to language models. However, this
  process is inherently challenging, as it requires both compressing visual
  signals into a compact representation and discretizing them into a fixed set of
  codes. Traditional discrete tokenizers typically learn the two tasks jointly,
  often leading to unstable training, low codebook utilization, and limited
  reconstruction quality. In this paper, we introduce
  CODA(COntinuous-to-Discrete Adaptation), a
  framework that decouples compression and discretization. Instead of training
  discrete tokenizers from scratch, CODA adapts off-the-shelf continuous VAEs --
  already optimized for perceptual compression -- into discrete tokenizers via a
  carefully designed discretization process. By primarily focusing on
  discretization, CODA ensures stable and efficient training while retaining the
  strong visual fidelity of continuous VAEs. Empirically, with 6
  times less training budget than standard VQGAN, our approach achieves a
  remarkable codebook utilization of 100% and notable reconstruction FID (rFID)
  of 0.43 and 1.34 for 8 times and 16 times
  compression on ImageNet 256times 256 benchmark.
paper_ai_summary: CODA adaptively transforms continuous VAEs into discrete tokenizers, achieving high codebook utilization and quality with significantly reduced training time.
paper_ai_keywords: discrete visual tokenizers token-based visual generation language models compression discretization continuous VAEs perceptual compression codebook utilization reconstruction FID ImageNet benchmark