title: "Principal Components" Enable A New Language of Images
summary: We introduce a novel visual tokenization framework that embeds a provable
  PCA-like structure into the latent token space. While existing visual
  tokenizers primarily optimize for reconstruction fidelity, they often neglect
  the structural properties of the latent space -- a critical factor for both
  interpretability and downstream tasks. Our method generates a 1D causal token
  sequence for images, where each successive token contributes non-overlapping
  information with mathematically guaranteed decreasing explained variance,
  analogous to principal component analysis. This structural constraint ensures
  the tokenizer extracts the most salient visual features first, with each
  subsequent token adding diminishing yet complementary information.
  Additionally, we identified and resolved a semantic-spectrum coupling effect
  that causes the unwanted entanglement of high-level semantic content and
  low-level spectral details in the tokens by leveraging a diffusion decoder.
  Experiments demonstrate that our approach achieves state-of-the-art
  reconstruction performance and enables better interpretability to align with
  the human vision system. Moreover, auto-regressive models trained on our token
  sequences achieve performance comparable to current state-of-the-art methods
  while requiring fewer tokens for training and inference.
paper_authors._id: 67d0f7032eaba9be7bf76e0e
paper_authors.name: Xin Wen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:39:00.455Z
paper_authors._id: 63483629ac5172169929da0e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665676793089-noauth.jpeg
paper_authors.fullname: Xin Wen
paper_authors.type: user
paper_authors.user: xwen99
paper_authors._id: 67d0f7032eaba9be7bf76e0f
paper_authors.name: Bingchen Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:38:56.945Z
paper_authors._id: 62dcd71075e9787ec5aa41ba
paper_authors.avatarUrl: /avatars/f37ce036b76180ed0fa004f9c8c09363.svg
paper_authors.fullname: Bingchen Zhao
paper_authors.type: user
paper_authors.user: tennant
paper_authors._id: 67d0f7032eaba9be7bf76e10
paper_authors.name: Ismail Elezi
paper_authors._id: 67d0f7032eaba9be7bf76e11
paper_authors.name: Jiankang Deng
paper_authors._id: 67d0f7032eaba9be7bf76e12
paper_authors.name: Xiaojuan Qi
paper_title: "Principal Components" Enable A New Language of Images
paper_summary: We introduce a novel visual tokenization framework that embeds a provable
  PCA-like structure into the latent token space. While existing visual
  tokenizers primarily optimize for reconstruction fidelity, they often neglect
  the structural properties of the latent space -- a critical factor for both
  interpretability and downstream tasks. Our method generates a 1D causal token
  sequence for images, where each successive token contributes non-overlapping
  information with mathematically guaranteed decreasing explained variance,
  analogous to principal component analysis. This structural constraint ensures
  the tokenizer extracts the most salient visual features first, with each
  subsequent token adding diminishing yet complementary information.
  Additionally, we identified and resolved a semantic-spectrum coupling effect
  that causes the unwanted entanglement of high-level semantic content and
  low-level spectral details in the tokens by leveraging a diffusion decoder.
  Experiments demonstrate that our approach achieves state-of-the-art
  reconstruction performance and enables better interpretability to align with
  the human vision system. Moreover, auto-regressive models trained on our token
  sequences achieve performance comparable to current state-of-the-art methods
  while requiring fewer tokens for training and inference.
paper_ai_keywords: visual tokenization framework PCA-like structure latent token space reconstruction fidelity structural properties interpretabiliy 1D causal token sequence explained variance principal component analysis salient visual features semantic-spectrum coupling effect diffusion decoder reconstruction performance human vision system auto-regressive models state-of-the-art methods