title: Adaptive Length Image Tokenization via Recurrent Allocation
summary: Current vision systems typically assign fixed-length representations to
  images, regardless of the information content. This contrasts with human
  intelligence - and even large language models - which allocate varying
  representational capacities based on entropy, context and familiarity. Inspired
  by this, we propose an approach to learn variable-length token representations
  for 2D images. Our encoder-decoder architecture recursively processes 2D image
  tokens, distilling them into 1D latent tokens over multiple iterations of
  recurrent rollouts. Each iteration refines the 2D tokens, updates the existing
  1D latent tokens, and adaptively increases representational capacity by adding
  new tokens. This enables compression of images into a variable number of
  tokens, ranging from 32 to 256. We validate our tokenizer using reconstruction
  loss and FID metrics, demonstrating that token count aligns with image entropy,
  familiarity and downstream task requirements. Recurrent token processing with
  increasing representational capacity in each iteration shows signs of token
  specialization, revealing potential for object / part discovery.
paper_authors._id: 672b801f26c6716f52485c26
paper_authors.name: Shivam Duggal
paper_authors._id: 672b801f26c6716f52485c27
paper_authors.name: Phillip Isola
paper_authors._id: 672b801f26c6716f52485c28
paper_authors.name: Antonio Torralba
paper_authors._id: 672b801f26c6716f52485c29
paper_authors.name: William T. Freeman
paper_title: Adaptive Length Image Tokenization via Recurrent Allocation
paper_summary: Current vision systems typically assign fixed-length representations to
  images, regardless of the information content. This contrasts with human
  intelligence - and even large language models - which allocate varying
  representational capacities based on entropy, context and familiarity. Inspired
  by this, we propose an approach to learn variable-length token representations
  for 2D images. Our encoder-decoder architecture recursively processes 2D image
  tokens, distilling them into 1D latent tokens over multiple iterations of
  recurrent rollouts. Each iteration refines the 2D tokens, updates the existing
  1D latent tokens, and adaptively increases representational capacity by adding
  new tokens. This enables compression of images into a variable number of
  tokens, ranging from 32 to 256. We validate our tokenizer using reconstruction
  loss and FID metrics, demonstrating that token count aligns with image entropy,
  familiarity and downstream task requirements. Recurrent token processing with
  increasing representational capacity in each iteration shows signs of token
  specialization, revealing potential for object / part discovery.
paper_ai_summary: A new method for learning variable-length token representations for 2D images through recursive processing and token specialization is proposed, validated by reconstruction loss and FID metrics.
paper_ai_keywords: variable-length token representations encoder-decoder architecture recurrent rollouts 2D tokens 1D latent tokens token specialization image entropy familiarity token count image compression reconstruction loss FID metrics