When Tokens Talk Too Much: A Survey of Multimodal Long-Context Token
  Compression across Images, Videos, and Audios
Multimodal large language models (MLLMs) have made remarkable strides,
largely driven by their ability to process increasingly long and complex
contexts, such as high-resolution images, extended video sequences, and lengthy
audio input. While this ability significantly enhances MLLM capabilities, it
introduces substantial computational challenges, primarily due to the quadratic
complexity of self-attention mechanisms with numerous input tokens. To mitigate
these bottlenecks, token compression has emerged as an auspicious and critical
approach, efficiently reducing the number of tokens during both training and
inference. In this paper, we present the first systematic survey and synthesis
of the burgeoning field of multimodal long context token compression.
Recognizing that effective compression strategies are deeply tied to the unique
characteristics and redundancies of each modality, we categorize existing
approaches by their primary data focus, enabling researchers to quickly access
and learn methods tailored to their specific area of interest: (1)
image-centric compression, which addresses spatial redundancy in visual data;
(2) video-centric compression, which tackles spatio-temporal redundancy in
dynamic sequences; and (3) audio-centric compression, which handles temporal
and spectral redundancy in acoustic signals. Beyond this modality-driven
categorization, we further dissect methods based on their underlying
mechanisms, including transformation-based, similarity-based, attention-based,
and query-based approaches. By providing a comprehensive and structured
overview, this survey aims to consolidate current progress, identify key
challenges, and inspire future research directions in this rapidly evolving
domain. We also maintain a public repository to continuously track and update
the latest advances in this promising area.
68881faaaf872d625c10c58b
Kele Shao
claimed_verified
2025-07-29T07:09:02.353Z
6696755fd26a65bd255184d3
/avatars/8d46c21a7b23f0100a7e3385fea61edf.svg
Kele Shao
user
keleshao
68881faaaf872d625c10c58c
Keda Tao
claimed_verified
2025-07-29T07:09:05.083Z
65ce28c6340c3e914285aa58
/avatars/ffaa6d6ce92274bff960f8ea229a37f8.svg
Keda TAO
user
KD-TAO
68881faaaf872d625c10c58d
Kejia Zhang
claimed_verified
2025-07-29T14:37:43.751Z
655469586bc4180700cf7a34
/avatars/252392d0c45783d8f149feac7a6215ec.svg
Kejia Zhang
user
KejiaRobust
68881faaaf872d625c10c58e
Sicheng Feng
claimed_verified
2025-07-29T12:51:07.500Z
67a4a26d5e65aa63c6d30e68
https://cdn-avatars.huggingface.co/v1/production/uploads/67a4a26d5e65aa63c6d30e68/GtodlJGw-_IL2DTXQTucz.jpeg
Sicheng Feng
user
FSCCS
68881faaaf872d625c10c58f
Mu Cai
68881faaaf872d625c10c590
Yuzhang Shang
68881faaaf872d625c10c591
Haoxuan You
68881faaaf872d625c10c592
Can Qin
68881faaaf872d625c10c593
Yang Sui
68881faaaf872d625c10c594
Huan Wang
When Tokens Talk Too Much: A Survey of Multimodal Long-Context Token
  Compression across Images, Videos, and Audios
Multimodal large language models (MLLMs) have made remarkable strides,
largely driven by their ability to process increasingly long and complex
contexts, such as high-resolution images, extended video sequences, and lengthy
audio input. While this ability significantly enhances MLLM capabilities, it
introduces substantial computational challenges, primarily due to the quadratic
complexity of self-attention mechanisms with numerous input tokens. To mitigate
these bottlenecks, token compression has emerged as an auspicious and critical
approach, efficiently reducing the number of tokens during both training and
inference. In this paper, we present the first systematic survey and synthesis
of the burgeoning field of multimodal long context token compression.
Recognizing that effective compression strategies are deeply tied to the unique
characteristics and redundancies of each modality, we categorize existing
approaches by their primary data focus, enabling researchers to quickly access
and learn methods tailored to their specific area of interest: (1)
image-centric compression, which addresses spatial redundancy in visual data;
(2) video-centric compression, which tackles spatio-temporal redundancy in
dynamic sequences; and (3) audio-centric compression, which handles temporal
and spectral redundancy in acoustic signals. Beyond this modality-driven
categorization, we further dissect methods based on their underlying
mechanisms, including transformation-based, similarity-based, attention-based,
and query-based approaches. By providing a comprehensive and structured
overview, this survey aims to consolidate current progress, identify key
challenges, and inspire future research directions in this rapidly evolving
domain. We also maintain a public repository to continuously track and update
the latest advances in this promising area.
A survey of token compression techniques for multimodal large language models, focusing on image, video, and audio data, categorizing methods by modality and mechanism.
multimodal large language models token compression self-attention mechanisms image-centric compression video-centric compression audio-centric compression transformation-based similarity-based attention-based query-based