title: TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming
    Videos
summary: The rapid growth of online video platforms, particularly live streaming
  services, has created an urgent need for real-time video understanding systems.
  These systems must process continuous video streams and respond to user queries
  instantaneously, presenting unique challenges for current Video Large Language
  Models (VideoLLMs). While existing VideoLLMs excel at processing complete
  videos, they face significant limitations in streaming scenarios due to their
  inability to handle dense, redundant frames efficiently. We introduce
  TimeChat-Online, a novel online VideoLLM that revolutionizes real-time video
  interaction. At its core lies our innovative Differential Token Drop (DTD)
  module, which addresses the fundamental challenge of visual redundancy in
  streaming videos. Drawing inspiration from human visual perception's Change
  Blindness phenomenon, DTD preserves meaningful temporal changes while filtering
  out static, redundant content between frames. Remarkably, our experiments
  demonstrate that DTD achieves an 82.8% reduction in video tokens while
  maintaining 98% performance on StreamingBench, revealing that over 80% of
  visual content in streaming videos is naturally redundant without requiring
  language guidance. To enable seamless real-time interaction, we present
  TimeChat-Online-139K, a comprehensive streaming video dataset featuring diverse
  interaction patterns including backward-tracing, current-perception, and
  future-responding scenarios. TimeChat-Online's unique Proactive Response
  capability, naturally achieved through continuous monitoring of video scene
  transitions via DTD, sets it apart from conventional approaches. Our extensive
  evaluation demonstrates TimeChat-Online's superior performance on streaming
  benchmarks (StreamingBench and OvOBench) and maintaining competitive results on
  long-form video tasks such as Video-MME and MLVU.
paper_authors._id: 680b24471c5fbd15909bf1f9
paper_authors.name: Linli Yao
paper_authors._id: 680b24471c5fbd15909bf1fa
paper_authors.name: Yicheng Li
paper_authors._id: 680b24471c5fbd15909bf1fb
paper_authors.name: Yuancheng Wei
paper_authors._id: 680b24471c5fbd15909bf1fc
paper_authors.name: Lei Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:13:14.592Z
paper_authors._id: 6038d6d0612f5eef3cc05ea9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6038d6d0612f5eef3cc05ea9/ryhvAX5djQpD5OrIlZQ1f.jpeg
paper_authors.fullname: Lei Li
paper_authors.type: user
paper_authors.user: tobiaslee
paper_authors._id: 680b24471c5fbd15909bf1fd
paper_authors.name: Shuhuai Ren
paper_authors._id: 680b24471c5fbd15909bf1fe
paper_authors.name: Yuanxin Liu
paper_authors._id: 680b24471c5fbd15909bf1ff
paper_authors.name: Kun Ouyang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-30T07:56:51.047Z
paper_authors._id: 62cd7aca7a036fc9941bb2b0
paper_authors.avatarUrl: /avatars/17a4d27af0243fd7dccf06066f671461.svg
paper_authors.fullname: kun ouyang
paper_authors.type: user
paper_authors.user: RUBBISHLIKE
paper_authors._id: 680b24471c5fbd15909bf200
paper_authors.name: Lean Wang
paper_authors._id: 680b24471c5fbd15909bf201
paper_authors.name: Shicheng Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T12:42:22.568Z
paper_authors._id: 64539588fc2b5f69e8faac76
paper_authors.avatarUrl: /avatars/111441eeb0dd4d8ad2f0d3f28277952a.svg
paper_authors.fullname: Li Shicheng
paper_authors.type: user
paper_authors.user: lscpku
paper_authors._id: 680b24471c5fbd15909bf202
paper_authors.name: Sida Li
paper_authors._id: 680b24471c5fbd15909bf203
paper_authors.name: Lingpeng Kong
paper_authors._id: 680b24471c5fbd15909bf204
paper_authors.name: Qi Liu
paper_authors._id: 680b24471c5fbd15909bf205
paper_authors.name: Yuanxing Zhang
paper_authors._id: 680b24471c5fbd15909bf206
paper_authors.name: Xu Sun
paper_title: TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming
    Videos
paper_summary: The rapid growth of online video platforms, particularly live streaming
  services, has created an urgent need for real-time video understanding systems.
  These systems must process continuous video streams and respond to user queries
  instantaneously, presenting unique challenges for current Video Large Language
  Models (VideoLLMs). While existing VideoLLMs excel at processing complete
  videos, they face significant limitations in streaming scenarios due to their
  inability to handle dense, redundant frames efficiently. We introduce
  TimeChat-Online, a novel online VideoLLM that revolutionizes real-time video
  interaction. At its core lies our innovative Differential Token Drop (DTD)
  module, which addresses the fundamental challenge of visual redundancy in
  streaming videos. Drawing inspiration from human visual perception's Change
  Blindness phenomenon, DTD preserves meaningful temporal changes while filtering
  out static, redundant content between frames. Remarkably, our experiments
  demonstrate that DTD achieves an 82.8% reduction in video tokens while
  maintaining 98% performance on StreamingBench, revealing that over 80% of
  visual content in streaming videos is naturally redundant without requiring
  language guidance. To enable seamless real-time interaction, we present
  TimeChat-Online-139K, a comprehensive streaming video dataset featuring diverse
  interaction patterns including backward-tracing, current-perception, and
  future-responding scenarios. TimeChat-Online's unique Proactive Response
  capability, naturally achieved through continuous monitoring of video scene
  transitions via DTD, sets it apart from conventional approaches. Our extensive
  evaluation demonstrates TimeChat-Online's superior performance on streaming
  benchmarks (StreamingBench and OvOBench) and maintaining competitive results on
  long-form video tasks such as Video-MME and MLVU.
paper_ai_summary: TimeChat-Online, an online VideoLLM with a Differential Token Drop module, efficiently handles real-time video streaming by reducing redundant frames, achieving superior performance compared to existing models.
paper_ai_keywords: Video Large Language Models Differential Token Drop Change Blindness phenomenon StreamingBench OvOBench Video-MME MLVU Proactive Response