EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
Prior efforts in light-weight model development mainly centered on CNN and
Transformer-based designs yet faced persistent challenges. CNNs adept at local
feature extraction compromise resolution while Transformers offer global reach
but escalate computational demands O(N^2). This ongoing trade-off
between accuracy and efficiency remains a significant hurdle. Recently, state
space models (SSMs), such as Mamba, have shown outstanding performance and
competitiveness in various tasks such as language modeling and computer vision,
while reducing the time complexity of global information extraction to
O(N). Inspired by this, this work proposes to explore the potential
of visual state space models in light-weight model design and introduce a novel
efficient model variant dubbed EfficientVMamba. Concretely, our EfficientVMamba
integrates a atrous-based selective scan approach by efficient skip sampling,
constituting building blocks designed to harness both global and local
representational features. Additionally, we investigate the integration between
SSM blocks and convolutions, and introduce an efficient visual state space
block combined with an additional convolution branch, which further elevate the
model performance. Experimental results show that, EfficientVMamba scales down
the computational complexity while yields competitive results across a variety
of vision tasks. For example, our EfficientVMamba-S with 1.3G FLOPs improves
Vim-Ti with 1.5G FLOPs by a large margin of 5.6% accuracy on ImageNet.
Code is available at: https://github.com/TerryPei/EfficientVMamba.
65f7ad0f933dd81304b79552
Xiaohuan Pei
admin_assigned
2024-03-18T09:07:34.813Z
6318661051edb569f1b4e68b
/avatars/9b98943bc609ca974abf0abff3e68633.svg
TerryPei
user
TerryPei
65f7ad0f933dd81304b79553
Tao Huang
65f7ad0f933dd81304b79554
Chang Xu
EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
Prior efforts in light-weight model development mainly centered on CNN and
Transformer-based designs yet faced persistent challenges. CNNs adept at local
feature extraction compromise resolution while Transformers offer global reach
but escalate computational demands O(N^2). This ongoing trade-off
between accuracy and efficiency remains a significant hurdle. Recently, state
space models (SSMs), such as Mamba, have shown outstanding performance and
competitiveness in various tasks such as language modeling and computer vision,
while reducing the time complexity of global information extraction to
O(N). Inspired by this, this work proposes to explore the potential
of visual state space models in light-weight model design and introduce a novel
efficient model variant dubbed EfficientVMamba. Concretely, our EfficientVMamba
integrates a atrous-based selective scan approach by efficient skip sampling,
constituting building blocks designed to harness both global and local
representational features. Additionally, we investigate the integration between
SSM blocks and convolutions, and introduce an efficient visual state space
block combined with an additional convolution branch, which further elevate the
model performance. Experimental results show that, EfficientVMamba scales down
the computational complexity while yields competitive results across a variety
of vision tasks. For example, our EfficientVMamba-S with 1.3G FLOPs improves
Vim-Ti with 1.5G FLOPs by a large margin of 5.6% accuracy on ImageNet.
Code is available at: https://github.com/TerryPei/EfficientVMamba.
The proposed EfficientVMamba combines visual state space models with convolutions to reduce computational complexity while maintaining competitive performance in vision tasks.
state space models SSM Mamba EfficientVMamba atrous-based selective scan skip sampling global representational features local representational features Efficient Visual State Space block convolution branch ImageNet FLOPs