title: VMamba: Visual State Space Model
summary: Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) stand as
  the two most popular foundation models for visual representation learning.
  While CNNs exhibit remarkable scalability with linear complexity w.r.t. image
  resolution, ViTs surpass them in fitting capabilities despite contending with
  quadratic complexity. A closer inspection reveals that ViTs achieve superior
  visual modeling performance through the incorporation of global receptive
  fields and dynamic weights. This observation motivates us to propose a novel
  architecture that inherits these components while enhancing computational
  efficiency. To this end, we draw inspiration from the recently introduced state
  space model and propose the Visual State Space Model (VMamba), which achieves
  linear complexity without sacrificing global receptive fields. To address the
  encountered direction-sensitive issue, we introduce the Cross-Scan Module (CSM)
  to traverse the spatial domain and convert any non-causal visual image into
  order patch sequences. Extensive experimental results substantiate that VMamba
  not only demonstrates promising capabilities across various visual perception
  tasks, but also exhibits more pronounced advantages over established benchmarks
  as the image resolution increases. Source code has been available at
  https://github.com/MzeroMiko/VMamba.
paper_authors._id: 65a9eb69cab1bfcc3afb8ff1
paper_authors.name: Yue Liu
paper_authors._id: 65a9eb69cab1bfcc3afb8ff2
paper_authors.name: Yunjie Tian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-19T13:44:43.598Z
paper_authors._id: 646736d376bb704aa1401dbf
paper_authors.avatarUrl: /avatars/302591dc0439eab4c9db2313b9bbf20a.svg
paper_authors.fullname: yunjie tian
paper_authors.type: user
paper_authors.user: sunsmarterjie
paper_authors._id: 65a9eb69cab1bfcc3afb8ff3
paper_authors.name: Yuzhong Zhao
paper_authors._id: 65a9eb69cab1bfcc3afb8ff4
paper_authors.name: Hongtian Yu
paper_authors._id: 65a9eb69cab1bfcc3afb8ff5
paper_authors.name: Lingxi Xie
paper_authors._id: 65a9eb69cab1bfcc3afb8ff6
paper_authors.name: Yaowei Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-19T13:47:44.503Z
paper_authors._id: 63e783b94577a86987a255bd
paper_authors.avatarUrl: /avatars/6ea80c7430e412a70c9cb8b80398624b.svg
paper_authors.fullname: yaowei wang
paper_authors.type: user
paper_authors.user: yaoweiwang
paper_authors._id: 65a9eb69cab1bfcc3afb8ff7
paper_authors.name: Qixiang Ye
paper_authors._id: 65a9eb69cab1bfcc3afb8ff8
paper_authors.name: Yunfan Liu
paper_title: VMamba: Visual State Space Model
paper_summary: Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) stand as
  the two most popular foundation models for visual representation learning.
  While CNNs exhibit remarkable scalability with linear complexity w.r.t. image
  resolution, ViTs surpass them in fitting capabilities despite contending with
  quadratic complexity. A closer inspection reveals that ViTs achieve superior
  visual modeling performance through the incorporation of global receptive
  fields and dynamic weights. This observation motivates us to propose a novel
  architecture that inherits these components while enhancing computational
  efficiency. To this end, we draw inspiration from the recently introduced state
  space model and propose the Visual State Space Model (VMamba), which achieves
  linear complexity without sacrificing global receptive fields. To address the
  encountered direction-sensitive issue, we introduce the Cross-Scan Module (CSM)
  to traverse the spatial domain and convert any non-causal visual image into
  order patch sequences. Extensive experimental results substantiate that VMamba
  not only demonstrates promising capabilities across various visual perception
  tasks, but also exhibits more pronounced advantages over established benchmarks
  as the image resolution increases. Source code has been available at
  https://github.com/MzeroMiko/VMamba.
paper_ai_summary: VMamba, a novel state space model architecture, combines global receptive fields and dynamic weights from ViTs with linear complexity, outperforming established models as image resolution increases.
paper_ai_keywords: Convolutional Neural Networks Vision Transformers global receptive fields dynamic weights state space model Visual State Space Model VMamba linear complexity Cross-Scan Module CSM spatial domain order patch sequences