title: VideoMamba: State Space Model for Efficient Video Understanding
summary: Addressing the dual challenges of local redundancy and global dependencies in
  video understanding, this work innovatively adapts the Mamba to the video
  domain. The proposed VideoMamba overcomes the limitations of existing 3D
  convolution neural networks and video transformers. Its linear-complexity
  operator enables efficient long-term modeling, which is crucial for
  high-resolution long video understanding. Extensive evaluations reveal
  VideoMamba's four core abilities: (1) Scalability in the visual domain without
  extensive dataset pretraining, thanks to a novel self-distillation technique;
  (2) Sensitivity for recognizing short-term actions even with fine-grained
  motion differences; (3) Superiority in long-term video understanding,
  showcasing significant advancements over traditional feature-based models; and
  (4) Compatibility with other modalities, demonstrating robustness in
  multi-modal contexts. Through these distinct advantages, VideoMamba sets a new
  benchmark for video understanding, offering a scalable and efficient solution
  for comprehensive video understanding. All the code and models are available at
  https://github.com/OpenGVLab/VideoMamba.
paper_authors._id: 65efca60e2f3a2d6588c8360
paper_authors.name: Kunchang Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-12T12:06:04.510Z
paper_authors._id: 61fb81006374891646732f37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1643872995181-61fb81006374891646732f37.jpeg
paper_authors.fullname: Kunchang Li
paper_authors.type: user
paper_authors.user: Andy1621
paper_authors._id: 65efca60e2f3a2d6588c8361
paper_authors.name: Xinhao Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:48:46.432Z
paper_authors._id: 64171b10f1e86908935a493a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64171b10f1e86908935a493a/tAwbHgTw714Vz-Qudz6Ij.jpeg
paper_authors.fullname: Xinhao Li
paper_authors.type: user
paper_authors.user: lixinhao
paper_authors._id: 65efca60e2f3a2d6588c8362
paper_authors.name: Yi Wang
paper_authors._id: 65efca60e2f3a2d6588c8363
paper_authors.name: Yinan He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-18T08:36:40.805Z
paper_authors._id: 62aafa49f29ff279b51f0182
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62aafa49f29ff279b51f0182/rQx8QFQGOY2qIhqJ8zSRj.jpeg
paper_authors.fullname: yinanhe
paper_authors.type: user
paper_authors.user: ynhe
paper_authors._id: 65efca60e2f3a2d6588c8364
paper_authors.name: Yali Wang
paper_authors._id: 65efca60e2f3a2d6588c8365
paper_authors.name: Limin Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-12T12:24:18.661Z
paper_authors._id: 62c77f4352d8ae531f5511f9
paper_authors.avatarUrl: /avatars/50198ccb02ccd286975a4613fbabee28.svg
paper_authors.fullname: Limin Wang
paper_authors.type: user
paper_authors.user: lmwang
paper_authors._id: 65efca60e2f3a2d6588c8366
paper_authors.name: Yu Qiao
paper_title: VideoMamba: State Space Model for Efficient Video Understanding
paper_summary: Addressing the dual challenges of local redundancy and global dependencies in
  video understanding, this work innovatively adapts the Mamba to the video
  domain. The proposed VideoMamba overcomes the limitations of existing 3D
  convolution neural networks and video transformers. Its linear-complexity
  operator enables efficient long-term modeling, which is crucial for
  high-resolution long video understanding. Extensive evaluations reveal
  VideoMamba's four core abilities: (1) Scalability in the visual domain without
  extensive dataset pretraining, thanks to a novel self-distillation technique;
  (2) Sensitivity for recognizing short-term actions even with fine-grained
  motion differences; (3) Superiority in long-term video understanding,
  showcasing significant advancements over traditional feature-based models; and
  (4) Compatibility with other modalities, demonstrating robustness in
  multi-modal contexts. Through these distinct advantages, VideoMamba sets a new
  benchmark for video understanding, offering a scalable and efficient solution
  for comprehensive video understanding. All the code and models are available at
  https://github.com/OpenGVLab/VideoMamba.
paper_ai_summary: VideoMamba, an adaptation of Mamba, addresses local and global challenges in video understanding through a linear-complexity operator, self-distillation, and compatibility with multiple modalities, setting a new benchmark.
paper_ai_keywords: Mamba VideoMamba 3D convolution neural networks video transformers linear-complexity operator self-distillation short-term actions long-term video understanding feature-based models multi-modal contexts