title: Images are Worth Variable Length of Representations
summary: Most existing vision encoders map images into a fixed-length sequence of
  tokens, overlooking the fact that different images contain varying amounts of
  information. For example, a visually complex image (e.g., a cluttered room)
  inherently carries more information and thus deserves more tokens than a simple
  image (e.g., a blank wall). To address this inefficiency, we propose DOVE, a
  dynamic vision encoder that produces a variable number of visual tokens (i.e.,
  continuous representation vectors) to reconstruct each image. Our results show
  that DOVE significantly reduces the average number of tokens while maintaining
  high reconstruction quality. In several linear probing and downstream
  multimodal tasks, it outperforms existing autoencoder-based tokenization
  methods when using far fewer tokens, capturing more expressive semantic
  features compared to fixed-length encoding. We further extend DOVE with
  query-conditioned tokenization. By guiding the model to focus on query-relevant
  regions, it achieves more efficient and targeted semantic extraction. Our code
  and checkpoints are available at https://dove-encoder.github.io/dove-encoder.
paper_authors._id: 6842d9cdc18e2d27141f82cd
paper_authors.name: Lingjun Mao
paper_authors._id: 6842d9cdc18e2d27141f82ce
paper_authors.name: Rodolfo Corona
paper_authors._id: 6842d9cdc18e2d27141f82cf
paper_authors.name: Xin Liang
paper_authors._id: 6842d9cdc18e2d27141f82d0
paper_authors.name: Wenhao Yan
paper_authors._id: 6842d9cdc18e2d27141f82d1
paper_authors.name: Zineng Tang
paper_title: Images are Worth Variable Length of Representations
paper_summary: Most existing vision encoders map images into a fixed-length sequence of
  tokens, overlooking the fact that different images contain varying amounts of
  information. For example, a visually complex image (e.g., a cluttered room)
  inherently carries more information and thus deserves more tokens than a simple
  image (e.g., a blank wall). To address this inefficiency, we propose DOVE, a
  dynamic vision encoder that produces a variable number of visual tokens (i.e.,
  continuous representation vectors) to reconstruct each image. Our results show
  that DOVE significantly reduces the average number of tokens while maintaining
  high reconstruction quality. In several linear probing and downstream
  multimodal tasks, it outperforms existing autoencoder-based tokenization
  methods when using far fewer tokens, capturing more expressive semantic
  features compared to fixed-length encoding. We further extend DOVE with
  query-conditioned tokenization. By guiding the model to focus on query-relevant
  regions, it achieves more efficient and targeted semantic extraction. Our code
  and checkpoints are available at https://dove-encoder.github.io/dove-encoder.
paper_ai_summary: A dynamic vision encoder, DOVE, generates a variable number of visual tokens for images, improving reconstruction quality and efficiency in multimodal tasks.
paper_ai_keywords: vision encoders tokens continuous representation vectors dynamic vision encoder reconstruction quality linear probing downstream multimodal tasks autoencoder-based tokenization query-conditioned tokenization semantic extraction