VideoMamba: Spatio-Temporal Selective State Space Model
<|reference_start|>VideoMamba: Spatio-Temporal Selective State Space Model: We introduce VideoMamba, a novel adaptation of the pure Mamba architecture, specifically designed for video recognition. Unlike transformers that rely on self-attention mechanisms leading to high computational costs by quadratic complexity, VideoMamba leverages Mamba's linear complexity and selective SSM mechanism for more efficient processing. The proposed Spatio-Temporal Forward and Backward SSM allows the model to effectively capture the complex relationship between non-sequential spatial and sequential temporal information in video. Consequently, VideoMamba is not only resource-efficient but also effective in capturing long-range dependency in videos, demonstrated by competitive performance and outstanding efficiency on a variety of video understanding benchmarks. Our work highlights the potential of VideoMamba as a powerful tool for video understanding, offering a simple yet effective baseline for future research in video analysis.<|reference_end|>
@article{park2024videomamba:,
    title={VideoMamba: Spatio-Temporal Selective State Space Model},
    author={Jinyoung Park, Hee-Seon Kim, Kangwook Ko, Minbeom Kim, Changick Kim},
    journal={arXiv preprint arXiv:2407.08476},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.08476},
    primaryClass={cs.CV}
}