title: SiLVR: A Simple Language-based Video Reasoning Framework
summary: Recent advances in test-time optimization have led to remarkable reasoning
  capabilities in Large Language Models (LLMs), enabling them to solve highly
  complex problems in math and coding. However, the reasoning capabilities of
  multimodal LLMs (MLLMs) still significantly lag, especially for complex
  video-language tasks. To address this issue, we present SiLVR, a Simple
  Language-based Video Reasoning framework that decomposes complex video
  understanding into two stages. In the first stage, SiLVR transforms raw video
  into language-based representations using multisensory inputs, such as short
  clip captions and audio/speech subtitles. In the second stage, language
  descriptions are fed into a powerful reasoning LLM to solve complex
  video-language understanding tasks. To handle long-context multisensory inputs,
  we use an adaptive token reduction scheme, which dynamically determines the
  temporal granularity with which to sample the tokens. Our simple, modular, and
  training-free video reasoning framework achieves the best-reported results on
  Video-MME (long), Video-MMMU (comprehension), Video-MMLU, CGBench, and EgoLife.
  Furthermore, our empirical study focused on video reasoning capabilities shows
  that, despite not being explicitly trained on video, strong reasoning LLMs can
  effectively aggregate multisensory input information from video, speech, and
  audio for complex temporal, causal, long-context, and knowledge acquisition
  reasoning tasks in video. Code is available at https://github.com/CeeZh/SILVR.
paper_authors._id: 683dfea7a6815c77ac987005
paper_authors.name: Ce Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T08:06:53.492Z
paper_authors._id: 6519fd9416f0c073c5c551c3
paper_authors.avatarUrl: /avatars/597ae2f69e1fa2b4bd32a69d1d999b55.svg
paper_authors.fullname: Ce Zhang
paper_authors.type: user
paper_authors.user: ceezh
paper_authors._id: 683dfea7a6815c77ac987006
paper_authors.name: Yan-Bo Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T08:47:27.971Z
paper_authors._id: 63a715c38729ce5b5640b80b
paper_authors.avatarUrl: /avatars/5dc0a167c5dae7756a77d79cfd6334b2.svg
paper_authors.fullname: yb
paper_authors.type: user
paper_authors.user: genjib
paper_authors._id: 683dfea7a6815c77ac987007
paper_authors.name: Ziyang Wang
paper_authors._id: 683dfea7a6815c77ac987008
paper_authors.name: Mohit Bansal
paper_authors._id: 683dfea7a6815c77ac987009
paper_authors.name: Gedas Bertasius
paper_title: SiLVR: A Simple Language-based Video Reasoning Framework
paper_summary: Recent advances in test-time optimization have led to remarkable reasoning
  capabilities in Large Language Models (LLMs), enabling them to solve highly
  complex problems in math and coding. However, the reasoning capabilities of
  multimodal LLMs (MLLMs) still significantly lag, especially for complex
  video-language tasks. To address this issue, we present SiLVR, a Simple
  Language-based Video Reasoning framework that decomposes complex video
  understanding into two stages. In the first stage, SiLVR transforms raw video
  into language-based representations using multisensory inputs, such as short
  clip captions and audio/speech subtitles. In the second stage, language
  descriptions are fed into a powerful reasoning LLM to solve complex
  video-language understanding tasks. To handle long-context multisensory inputs,
  we use an adaptive token reduction scheme, which dynamically determines the
  temporal granularity with which to sample the tokens. Our simple, modular, and
  training-free video reasoning framework achieves the best-reported results on
  Video-MME (long), Video-MMMU (comprehension), Video-MMLU, CGBench, and EgoLife.
  Furthermore, our empirical study focused on video reasoning capabilities shows
  that, despite not being explicitly trained on video, strong reasoning LLMs can
  effectively aggregate multisensory input information from video, speech, and
  audio for complex temporal, causal, long-context, and knowledge acquisition
  reasoning tasks in video. Code is available at https://github.com/CeeZh/SILVR.
paper_ai_summary: SiLVR, a language-based framework, enhances multimodal LLMs' video reasoning by leveraging adaptive token reduction, achieving top results on several benchmarks.
paper_ai_keywords: Large Language Models multimodal LLMs video-language tasks Language-based Video Reasoning multisensory inputs adaptive token reduction temporal granularity Video-MME Video-MMMU Video-MMLU CGBench EgoLife