title: BIMBA: Selective-Scan Compression for Long-Range Video Question
    Answering
summary: Video Question Answering (VQA) in long videos poses the key challenge of
  extracting relevant information and modeling long-range dependencies from many
  redundant frames. The self-attention mechanism provides a general solution for
  sequence modeling, but it has a prohibitive cost when applied to a massive
  number of spatiotemporal tokens in long videos. Most prior methods rely on
  compression strategies to lower the computational cost, such as reducing the
  input length via sparse frame sampling or compressing the output sequence
  passed to the large language model (LLM) via space-time pooling. However, these
  naive approaches over-represent redundant information and often miss salient
  events or fast-occurring space-time patterns. In this work, we introduce BIMBA,
  an efficient state-space model to handle long-form videos. Our model leverages
  the selective scan algorithm to learn to effectively select critical
  information from high-dimensional video and transform it into a reduced token
  sequence for efficient LLM processing. Extensive experiments demonstrate that
  BIMBA achieves state-of-the-art accuracy on multiple long-form VQA benchmarks,
  including PerceptionTest, NExT-QA, EgoSchema, VNBench, LongVideoBench, and
  Video-MME. Code, and models are publicly available at
  https://sites.google.com/view/bimba-mllm.
paper_authors._id: 67d2f46bbc86adc45acd3888
paper_authors.name: Md Mohaiminul Islam
paper_authors._id: 67d2f46bbc86adc45acd3889
paper_authors.name: Tushar Nagarajan
paper_authors._id: 67d2f46bbc86adc45acd388a
paper_authors.name: Huiyu Wang
paper_authors._id: 67d2f46bbc86adc45acd388b
paper_authors.name: Gedas Bertasius
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:25:32.212Z
paper_authors._id: 65d60993c2c6418adc99206a
paper_authors.avatarUrl: /avatars/fb1f9d5af7022bddf4cb090cd477108d.svg
paper_authors.fullname: Gedas Bertasius
paper_authors.type: user
paper_authors.user: gberta
paper_authors._id: 67d2f46bbc86adc45acd388c
paper_authors.name: Lorenzo Torresani
paper_title: BIMBA: Selective-Scan Compression for Long-Range Video Question
    Answering
paper_summary: Video Question Answering (VQA) in long videos poses the key challenge of
  extracting relevant information and modeling long-range dependencies from many
  redundant frames. The self-attention mechanism provides a general solution for
  sequence modeling, but it has a prohibitive cost when applied to a massive
  number of spatiotemporal tokens in long videos. Most prior methods rely on
  compression strategies to lower the computational cost, such as reducing the
  input length via sparse frame sampling or compressing the output sequence
  passed to the large language model (LLM) via space-time pooling. However, these
  naive approaches over-represent redundant information and often miss salient
  events or fast-occurring space-time patterns. In this work, we introduce BIMBA,
  an efficient state-space model to handle long-form videos. Our model leverages
  the selective scan algorithm to learn to effectively select critical
  information from high-dimensional video and transform it into a reduced token
  sequence for efficient LLM processing. Extensive experiments demonstrate that
  BIMBA achieves state-of-the-art accuracy on multiple long-form VQA benchmarks,
  including PerceptionTest, NExT-QA, EgoSchema, VNBench, LongVideoBench, and
  Video-MME. Code, and models are publicly available at
  https://sites.google.com/view/bimba-mllm.
paper_ai_summary: BIMBA, an efficient state-space model using the selective scan algorithm, improves accuracy on long-form VQA benchmarks by selecting critical information and reducing token sequences for LLM processing.
paper_ai_keywords: self-attention mechanism spatiotemporal tokens selective scan algorithm state-space model long-form VQA LLM processing PerceptionTest NExT-QA EgoSchema VNBench LongVideoBench Video-MME