## Cross-Scan Mamba with Masked Training for Robust Spectral Imaging

Wenzhe Tian 1 , ∗ , Haijin Zeng 2 , * , Yin-Ping Zhao 3 , Yongyong Chen 4 , †

, Zhen Wang 3 , Xuelong Li 3 1 University of Helsinki, 2 IMEC &amp; Universiteit Gent, Harbin Institute of Technology (Shenzhen)

3 Northwestern Polytechnical University, 4

## Abstract

Snapshot Compressive Imaging (SCI) enables fast spectral imaging but requires effective decoding algorithms for hyperspectral image (HSI) reconstruction from compressed measurements. Current CNN-based methods are limited in modeling long-range dependencies, while Transformerbased models face high computational complexity. Although recent Mamba models outperform CNNs and Transformers in RGB tasks concerning computational efficiency or accuracy, they are not specifically optimized to fully leverage the local spatial and spectral correlations inherent in HSIs. To address this, we propose the C rossS canning Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local balanced context encoding and cross-channel interaction promotion. Besides, while current reconstruction algorithms perform increasingly well in simulation scenarios, they exhibit suboptimal performance on real data due to limited generalization capability. During the training process, the model may not capture the inherent features of the images but rather learn the parameters to mitigate specific noise and loss, which may lead to a decline in reconstruction quality when faced with real scenes. To overcome this challenge, we propose a masked training method to enhance the generalization ability of models. Experiment results show that our CS-Mamba achieves state-of-the-art performance and the masked training method can better reconstruct smooth features to improve the visual quality.

## 1. Introduction

Hyperspectral images (HSIs) contain a large number of spectral bands to store rich detail information and thus are widely used in various fields such as object tracking [25, 36, 40], remote sensing [9, 21, 49], super-resolution [12, 20, 44], and medical image processing [1, 30, 33]. Traditional imaging systems used for collecting HSIs involve scanning along the spatial or spectral dimensions but this imaging pro-

* Equal Contribution, † Corresponding Author



Figure 1. PSNR-FLOPs comparisons of CS-Mamba and previous Deep Unfolding SOTA methods.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-d3d9d159-7a1a-42a0-a08f-61b16be304f3.jpg)

cess is time-consuming and can not effectively capture dynamic objects. Recently, snapshot compressive imaging (SCI) systems have been developed [29, 41] as an alternative method for obtaining HSIs at video rate. SCI systems capture multiple image bands in a single snapshot, compressing the information of HSIs into a 2D measurement. [6, 15, 42, 51]. Among these systems, Coded Aperture Snapshot Spectral Imaging (CASSI) has emerged and become mainstream [48, 50]. CASSI utilizes a physical mask to modulate signals from HSIs, shifting these modulated images to different spatial locations on the detector plane via a disperser [3, 47]. However, the performance of the CASSI system heavily depends on the efficiency of the reconstruction algorithm.

In recent years, numerous reconstruction algorithms have been proposed, including model-based methods and deep learning-based methods. Traditional model-based methods typically attempt to model the inverse process of image reconstruction, using manually designed priors to derive the solution to an optimization problem. Although these methods are highly interpretable, they often require manual parameter tuning and tend to be slow. With the success of deep learning methods in various visual tasks, CNN and Transformer-

based methods have been developed and applied to HSI reconstruction, achieving higher image reconstruction quality and speed than traditional algorithms. The CNN-based methods [32, 35] have an advantage in extracting local features from images, but they lack the ability to model long-range dependencies effectively. Transformer-based methods [2-5] realize a global receptive field, resulting in superior performance compared to CNNs but higher computational costs. Although some mechanisms can mitigate this issue, they do not fundamentally change the quadratic complexity problem caused by the attention mechanism in the Transformer architecture.

The limitation of Transformers in spectral SCI remains unresolved. However, in the realm of large language model and RGB image vision, an advanced solution for this problem exists, known as Mamba [16]. Through data-dependent selective SSMs with parallel scanning and hardware-wise strategies, Mamba establishes a global receptive field and achieves near-linear complexity. Mamba-based methods have been shown to outperform previous methods in various tasks due to their excellent long sequence modeling capabilities and lower computational costs [19, 27, 45, 54].

Subsequently, many works have begun to explore the application and performance of the Mamba model in various visual tasks. Representative works, such as the visual state space model [39] and vision Mamba [53], have demonstrated Mamba's superiority for image processing, particularly in obtaining fewer parameters and faster speed. However, most Mamba-based models only conduct the scanning process along the spatial dimension, lacking cross-channel interaction, despite the importance of spectral information for HSI reconstruction. Recently, some Mamba-based methods have made progress in developing an SSM mechanism along the channel (spectral) dimension for certain visual tasks [22, 43], achieving better results compared to scanning only along the spatial dimension. However, this scanning method does not consider the inherently local features of HSIs in both spatial and spectral dimensions. Additionally, previous reconstruction algorithms have achieved better results in experiments by continuously improving the backbone and framework. However, it is often overlooked that achieving superior performance in the simulation experiment is not entirely linearly correlated with achieving better visual effects in the real experiment. Simulation experiments primarily address the losses caused by the reconstruction compression process, whereas real experiments introduce noise that simulates real-world conditions, thus requiring the model to possess stronger generalization capabilities.

However, the degradation caused by insufficient generalization capability is, in fact, not easy to notice. As a result, most current work has not recognized this issue. This oversight is mainly due to the limited test data from real-world experiments, which makes certain issues less likely to be no-

Figure 2. Schematic of the CASSI system: A 3D HSI cube is modulated by a mask, sheared by a disperser, and transformed into a 2D-coded measurement.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-fdd7751a-b7a4-4734-9c44-b3ab03b826ac.jpg)

ticed. In most scenes, an improvement in quantitative results in simulation experiments is indeed associated with better performance in real scenes, which has led to some confusion. In this paper, we provide a detailed explanation of this phenomenon, making the problem more apparent. Notably, certain optimization strategies commonly used in previous simulation experiments can even deteriorate specific features in real-world scenes. We assume this phenomenon arises due to the model's overemphasizing of compression loss learning, thereby neglecting other noise (like shot noise) and the semantic information inherent in images. Experimental results in this paper confirm this point.

In this paper, we propose CS-Mamba, a spatial-spectral SSM with our cross-scanning method and local enhancement, to improve HSI reconstruction in SCI. Our CS-Mamba aims to strike a balance between global and local information utilization in HSIs. CS-Mamba comprises two main components: spatial SSM and spectral-spatial cross SSM. The spatial SSM employs a combination of global and local scanning methods, enhancing row-by-row scanning with both global and local modeling. Meanwhile, the spectral-spatial cross SSM incorporates an innovative across-scanning mechanism, leveraging spatial-spectral local cubes to enhance the module's ability to capture local details. Besides, we introduce a masked training strategy in real scenarios. This strategy forces the model to learn the inherent features of images, enhancing the generalization capabilities of previous methods and thereby achieving better results in real scenes. Overall, our contributions are summarized as follows:

- · We present a novel spectral-spatial state space model for HSI reconstruction, which can simultaneously integrate information from both spectral and spatial dimensions to adequately model spatial context and spectral features.
- · We propose a cross-scanning method that integrates local features from both spatial and spectral dimensions within a single scan. Additionally, we introduce a local scanning approach within the spatial SSM to further enhance local feature extraction.
- · We introduce a masked training strategy to promote the robustness of models in real scenarios.

## 2. Preliminaries and Related Works

CASSI System. An imaging process CASSI system is illustrated in Fig. 2. The 3D HSI data cube is denoted as

F ∈ R H × W N × λ where H, W, and N λ represent the height, width, and number of wavelengths of the 3D HSI, respectively. A physical mask M ∈ R H × W is used to modulate the 3D HSI F as: F ′ (: , : , n λ ) = F ′ (: , : , n λ ) ⊙ M , where ⊙ represents the element-wise product and F ′ ∈ R H × W N × λ is the modulated 3D HSIs. After the modulated cube passes the disperser, F ′ is tilted and considered to be sheared along the y-axis. We use F ′′ ∈ R H × ( W d N + ( λ -1)) × N λ to denote the tilted cube and assume n λ to be the reference wavelength. That is, image F ′ (: , : , n λ c ) is not sheared along the y-axis. Then we have: F ′′ ( u, v, n λ ) = F ′ ( x, y + ( d λ n -λ c ) , n λ ) , where ( u, v ) indicates the coordinate system on the detector plane, and λ n is the wavelength of channel n λ . Here, d λ ( n -λ c ) signifies the spatial shifting for band n λ . Finally, the measurement Y ∈ R H × [ W d N + ( λ -1)] can be obtained by

$$\mathbf Y = \sum _ { n _ { \lambda } = 1 } ^ { N _ { \lambda } } \mathbf F ^ { ^ { \prime \prime } } ( \colon, \colon, n _ { \lambda } ) + \mathbf N, \quad \quad ( 1 ) \quad \text{fid} \\ \quad \quad \quad$$

where N ∈ R H × [ W d N + ( λ -1)] represents the noise of CASSI caused by the sensing detector.

State Space Models. Recent advancements in state space models (SSMs) have spurred the development of various innovative methods and architectures. Notably, the structured state space model (S4) [18] has emerged as a pioneering approach due to its effectiveness in capturing long-range dependencies. S4 maps a 1D sequence x t ( ) ∈ R → y t ( ) ∈ R by an implicit latent state h t ( ) ∈ R N and decodes it into an output sequence, which can be formulated as:

$$& h ^ { \prime } ( t ) = \mathbf A h ( t ) + \mathbf B x ( t ), \\ & y ( t ) = \mathbf C h ( t ) + \mathbf D x ( t ),$$

where N is the state size, A ∈ R N × N , B ∈ R N × 1 , C ∈ R 1 × N , and D ∈ R . After that, a discretization process is adopted to integrate Eq. (2) into the deep learning algorithm, using the zero-order hold (ZOH) rule defined as,

$$\overline { \mathbf A } & = \exp ( \Delta \mathbf A ), \\ \overline { \mathbf B } & = ( \Delta \mathbf A ) ^ { - 1 } ( \exp ( \mathbf A ) - \mathbf I ) \cdot \Delta \mathbf B, \quad \ \ \ \ ( 3 ) \quad \ \ \ \$$

where A and B represent the discretized forms of the parameters A and B , respectively. ∆ is denoted as the timescale parameter to transform the continuous parameters A and B into discrete ones.

Mamba in Vision. Selective structured state space models (S6), or Mamba [17], use a selective mechanism for datadependent dynamics, enhancing language and visual task performance. Vision Mamba (Vim) [53] addresses Mamba's unidirectional modeling by adding bidirectional SSMs for global context and position embeddings for visual recognition. VMamba [28] bridges 1-D and 2-D scanning via the Cross-Scan Module (CSM). CU-Mamba [11] adds Channel

SSM to capture channel correlations, while S 2 Mamba [43] introduces spectral scanning for semantic retrieval in HSI data. However, these methods overlook local similarity in both spatial and spectral dimensions of HSIs.

## 3. The Proposed CS-Mamba Method

## 3.1. Architecture of CS-Mamba

The architecture of our CS-Mamba is illustrated in Fig. 3. CS-Mamba leverages a deep unfolding framework guided by mask degradation patterns. Using MAP theory, we model the HSI reconstruction process as follows:

$$\hat { x } = \arg \min _ { x } \ \frac { 1 } { 2 } | | y - \Phi x | | ^ { 2 } + \tau R ( x ), \quad ( 4 )$$

where || y -Φx || 2 is the data fidelity term, R ( x ) represents the image prior, and τ controls the balance between the data fidelity and prior terms.

HQS introduces an auxiliary variable z , resulting in a constrained optimization problem given by

$$\hat { x } = \arg \min _ { x } \frac { 1 } { 2 } \| y - \Phi x \| ^ { 2 } + \lambda R ( z ) \ \ s. t. \ \ z = x. \ \ ( 5 )$$

Eq. (5) is solved by minimizing

$$\underset { \infty } { g } \quad \mathcal { L } _ { \mu } ( x, z ) = \frac { 1 } { 2 } \| y - \Phi x \| ^ { 2 } + \lambda R ( z ) + \frac { \mu } { 2 } \| z - x \| ^ { 2 }, \ \ ( 6 )$$

where µ is a penalty parameter. Subsequently, Eq. (6) can be addressed by iteratively solving a data subproblem (xsubproblem) and a prior subproblem (z-subproblem):

$$x _ { k } = \arg \min _ { x } \| y - \Phi x \| ^ { 2 } + \mu \| x - z _ { k - 1 } \| ^ { 2 }, \quad ( 7 a )$$

$$z _ { k } = \arg \min _ { z } \frac { 1 } { 2 ( \sqrt { \lambda / \mu } ) ^ { 2 } } \| z - x _ { k } \| ^ { 2 } + R ( z ). \quad ( 7 b )$$

The data subproblem, Eq. (7a), usually has a fast closedform solution as

$$x _ { k } = ( \Phi ^ { \top } \Phi + \mu I ) ^ { - 1 } ( \Phi ^ { \top } y + \mu z _ { k - 1 } ), \quad ( 8 )$$

where I is an identity matrix, the matrix inversion in Eq. (8) can be written as

$$\text{aram} \quad ( \Phi ^ { \top } \Phi + \mu I ) ^ { - 1 } = \mu ^ { - 1 } I - \mu ^ { - 1 } \Phi ^ { \top } ( I + \Phi \mu ^ { - 1 } \Phi ^ { \top } ) ^ { - 1 } \Phi \mu ^ { - 1 }. \\ \text{scale} \quad \text{$\Phi$} \quad \text{$\Phi$} \quad \text{$\Phi$} \quad \text{$\Phi$} \quad \text{$\Phi$} \quad \text{$\Phi$} \quad \text{$\Phi$}.$$

In CASSI systems, Φ ∈ R HW ′ × HW N ′ λ is a block diagonal matrix, thus, ΦΦ ⊤ is a diagonal matrix that can be defined as

$$\Phi \Phi ^ { \top } \stackrel { d e f } { = } \text{diag} \{ \phi _ { 1 }, \dots, \phi _ { i }, \dots, \phi _ { H W ^ { \prime } } \}. \quad ( 1 0 )$$

By plugging ΦΦ ⊤ into ( I +Φ µ -1 Φ ) ⊤ -1 , we obtain:

$$\underset { \text{ia the} } { \underset { \text{ia the} } { \text{ogni} } } \quad ( I + \Phi \mu ^ { - 1 } \Phi ^ { \top } ) ^ { - 1 } = \text{diag} \{ \frac { \mu } { \mu + \phi _ { 1 } }, \dots, \frac { \mu } { \mu + \phi _ { i } }, \dots, \frac { \mu } { \mu + \phi _ { H W ^ { \prime } } } \}.$$

Figure 3. The flowchart of the proposed CS-Mamba . (a) Parameter Estimation Network. This module is used for capturing parameter α and noise level β to guide the reconstruction process. (b) The U-shaped denoiser network and DUN framework. (c) The Spatial-Spectral SSM Block adopted in (b). This module mainly consists of a spatial SSM and a spectral SSM, depicted in the sub-flowchart (d) and (e), respectively. (f) The specific method of LE-SSM in the Spatial SSM Module. Scanning directions are divided into global and local branches. (g) The illustration of our Cross-Scan Mechanism. The scanning process is demonstrated by labeling the order of each data point.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-184f8be2-0b60-4956-9955-418e9a224397.jpg)

By plugging Eq. (9), Eq. (10) and Eq. (11) into Eq. (8) and simplifying the formula, we have

$$x _ { k } = z _ { k - 1 } + \Phi ^ { \top } [ ( y - \Phi z _ { k - 1 } ). / ( \mu + d i a g ( \Phi \Phi ^ { \top } ) ) ], \, ( 1 2 ) \quad \text{ and } \varepsilon$$

where diag represents the extraction of elements from its diagonal to form a vector.

For the denoiser network, we employ a customized Mamba using a spatial-spectral SSM block as its fundamental unit. The architecture comprises an encoder, a bottleneck, and a decoder. Given a measurement y ∈ R H W ( + ( d N λ -1)) , we reverse the dispersion and shift back Y to initialize the input signal X ∈ R H × W N × λ :

The prior subproblem Eq. (7b), from a Bayesian perspective, corresponds to Gaussian denoising on x k with noise level √ λ/µ . To address this, Eq. (7b) could be rewritten as,

$$z _ { k } = D e n o i s e r ( x _ { k }, \sqrt { \lambda / \mu } ). \quad \ \ ( 1 3 ) \quad \ \text{to}$$

DUNs iteratively solve the data subproblem (Eq. 12) and the prior subproblem (Eq. 13) to address ill-posed inverse problems. The data subproblem depends on the degradation matrix, which can be estimated using either a sensing matrix [2, 3] or a DNN [5]. However, sensing matrices may not accurately reflect real-world degradations, and directly modeling them is challenging. The prior subproblem is treated as a denoising task, but the noise level is unknown in reconstruction, affecting performance. Following [14], we iteratively update the degradation matrix and parameter estimates, learning constant noise and regularization parameters towards each stage due to computational cost, as detailed in the supplementary material.

$$\mathbf X ( x, y, n _ { \lambda } ) = \mathbf Y ( x, y - d ( \lambda _ { n } - \lambda _ { c } ) ). \quad ( 1 4 )$$

We first concatenate the input X 0 with the physical mask Y ∈ R H × W N × λ and then employ a 1 × 1 convolution layer to integrate information from the original signal and the physical mask. Moreover, the integrated input X 0 undergoes a 3 × 3 convolution embedding into a feature X ′ 0 ∈ R H × W C × . X ′ 0 undergoes downsampling and upsampling before entering the decoder, followed by a 3 × 3 convolution to extract residual information. Summing the integrated input and residual output yields the reconstructed HSIs. We adjust the number of spatial-spectral SSM blocks in the encoder, bottleneck, and decoder to create different CS-Mamba model sizes.

## 3.2. Enhanced Local SSM (LE-SSM)

Although Mamba effectively models long-range dependencies, directly using the SSM mechanism often limits its ability to capture local features. Recent methods integrate

Ground Truth CS-Mamba w/o mask CS-Mamba w/ mask Figure 4. A visual comparison of with and w/o masked methods in our CS-Mamba model on real scene 1.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-1639adc2-7c19-4372-aa61-9e456f63a109.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-4cf08d35-73d0-4e18-8594-af25fab30dd8.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-4c40879b-e280-46fc-a371-c91192373169.jpg)

Table 1. Comparisons of our denoiser with previous End-to-End networks on PSNR, SSIM, FLOPs, Params, and Inference Time on the test dataset.

| Framework         |   MST-S [3] |   CST-S [2] |   Cross-Scan SSM |
|-------------------|-------------|-------------|------------------|
| PSNR              |      34.14  |      34.53  |           34.74  |
| SSIM              |       0.938 |       0.945 |            0.946 |
| Params (M)        |       0.93  |       1.2   |            0.81  |
| FLOPS (G)         |      12.96  |      11.67  |            7.01  |
| Inference Time(s) |       0.022 |       0.041 |            0.009 |

CNNs into the network to leverage their local feature extraction capabilities, enhancing original SSM performance. However, this increases the parameter count, undermining the lightweight nature of SSMs. While adding CNN can boost performance, it does not address the core issue. Inspired by LocalMamba [24], we propose a simplified local mechanism in the spectral dimension. This involves using two global scanning directions, traversing the image row by row in both forward and backward directions, while the other two scanning directions serve as local branches to unfold and scan local patches.

The specific details of the scanning process are illustrated in Fig. 3(d). Given the input X ∈ R H × W C × , we first generate four copies of input data with different arrangements. For the global branch, we directly flatten the feature map into X 1 ∈ R L × C , where L = H × W , and then reverse it to generate X 2 . For the local branch, we first set the size of our patch flattening each patch into a 1D sequence, and then linking each patch together to generate X 3 . Similarly, we reverse X 3 to obtain X 4 . After the generation of input data in four directions, we conduct the process as follows:

$$& X ^ { ^ { \prime } } = C o n c a t ( X _ { 1 }, X _ { 2 }, X _ { 3 }, X _ { 4 } ), \\ & \hat { X } = L E \text{-} S S M ( X ^ { ^ { \prime } } ).$$

The structure of LE-SSM is shown in Fig. 3(b). It is worth noting that our method does not employ any CNNs or spatial and channel attention mechanisms based on MLPs, ensuring that our method does not add any extra parameters or time consumption compared to the global cross-scan mechanism [28]. However, it also means we need to carefully choose the size of local patches, or else the reconstruction result will even become worse. In our implementation, we use patches of size 4 × 4 to achieve a relatively better performance.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-38f2d46b-8067-4230-a95c-0890922e7641.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-cf39e1db-0b5a-4b4e-8baa-e3095036d2a9.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-b289eb70-66a1-4242-b253-5641c2445153.jpg)

DAUHST

RGB ref.

Measurement

Figure 5. A visual comparison of the performance of MST, CST, and DAUHST methods in the real scene.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-0880ebdf-77eb-430a-b77a-e2d6a838b9f0.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-ff3850b7-a9c4-4f7a-a5a5-d6d62c2aeb1e.jpg)

## 3.3. Cross-Scanning SSM (CS-SSM) Mechanism

Our LE-SSM mechanism creates a locally enhanced global receptive field but does not fully utilize the rich spectral information in HSIs. Recent methods [11, 43] have integrated channel information by scanning along the spectral dimension; however, this approach overlooks local features in both spectral and spatial dimensions. Neighboring pixels and bands often share similarities, but direct spectral scanning can increase the distance between highly correlated data points, reducing reconstruction quality.

Table 2. Comparisons of our CS-Mamba-S (9stg) with previous Transformer-based methods on PSNR, SSIM, FLOPs, and Params.

| Framework   | DAUHST [5]   | DERNN [14]   | CS-Mamba-S   |
|-------------|--------------|--------------|--------------|
| PSNR        | 38.36        | 39.93        | 40.00        |
| SSIM        | 0.967        | 0.976        | 0.980        |
| Params (M)  | 6.15M        | 0.65M        | 0.57M        |
| FLOPS (G)   | 79.50        | 81.99        | 65.77        |

To address the above issues, we propose a local scanning method that integrates local features from both spatial and spectral dimensions. This mechanism attempts to exploit the local properties of HSIs in both spatial and spectral dimensions while leveraging the rich spectral information to guide the reconstruction process. First, we rearrange the input X ∈ R H × W C × into patches along the spatial context by dividing the input HSIs into a series of smaller data cubes S ∈ R P × × P C , which will be flattened into sequences for scanning. After that, we organize the scanning order within each sequence based on our spatial-spectral local cubes C ∈ R h × × w c . As shown in Fig. 3(e), different colors indicate each spatial-spectral local cube. The scanning order of each data point is also depicted. The process can be formulated as follows:

$$& \mathbf C = D i v i d e ( \mathbf X ), \\ & \mathbf C ^ { ^ { \prime } } = C S \text{-} S S M ( \mathbf C ),$$

where C ′ denotes the output patch of CS-SSM. After the SSM mechanism, we reorganize the patches back into the same order as the input X . By scanning the data points within each data cube, we can leverage the local similarity between adjacent spectral bands and pixels, thus achieving better results than previous methods that directly scan along the spectral dimension.

## 3.4. Masked Training Strategy

Before presenting our masked training strategy, we discuss our observed phenomena and motivation for this approach.

Table 3. Comparisons between CS-Mamba and SOTA methods on 10 simulation scenes (S1 ∼ S10). Params, FLOPs, PSNR (upper entry in each cell), and SSIM (lower entry in each cell) are reported. The best and second-best results are highlighted.

| Algorithms         | Reference   | Params   |   GFLOPs | S1          | S2          | S3          | S4          | S5          | S6          | S7          | S8          | S9          | S10         | Avg         |
|--------------------|-------------|----------|----------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|
| DIP-HSI [34]       | ICCV 2021   | 33.85M   |    64.42 | 31.32 0.855 | 25.89 0.699 | 29.91 0.839 | 38.69 0.926 | 27.45 0.796 | 29.53 0.824 | 27.46 0.700 | 27.69 0.802 | 33.46 0.863 | 26.10 0.733 | 29.75 0.803 |
| ADMM-Net [31]      | ICCV 2019   | 4.27M    |    78.58 | 34.03 0.919 | 33.57 0.904 | 34.82 0.933 | 39.46 0.971 | 31.83 0.924 | 32.47 0.926 | 32.01 0.898 | 30.49 0.907 | 33.38 0.917 | 30.55 0.899 | 33.26 0.920 |
| DGSMP [23]         | TPAMI 2023  | 3.76M    |   646.65 | 33.26 0.915 | 32.09 0.898 | 33.06 0.925 | 40.54 0.964 | 28.86 0.882 | 33.08 0.937 | 30.74 0.886 | 31.55 0.923 | 31.66 0.911 | 31.44 0.925 | 32.63 0.917 |
| DiffSCI [37]       | CVPR 2024   | 93.52M   |   193.76 | 34.96 0.907 | 34.60 0.905 | 39.83 0.949 | 42.65 0.951 | 35.21 0.946 | 33.12 0.917 | 36.29 0.944 | 30.42 0.887 | 37.27 0.931 | 28.49 0.821 | 35.28 0.916 |
| MST-L [3]          | CVPR 2022   | 2.03M    |    28.15 | 35.28 0.946 | 35.93 0.948 | 36.19 0.955 | 41.64 0.977 | 32.83 0.950 | 34.61 0.957 | 33.95 0.932 | 32.80 0.953 | 34.86 0.947 | 32.66 0.946 | 35.07 0.951 |
| CST-L+ [2]         | ECCV 2022   | 3.00M    |    40.1  | 35.64 0.951 | 36.79 0.957 | 37.71 0.965 | 41.38 0.981 | 32.95 0.957 | 35.58 0.966 | 34.54 0.947 | 34.07 0.964 | 35.62 0.959 | 32.82 0.949 | 35.71 0.960 |
| DAUHST-9stg [5]    | NIPS 2022   | 6.15M    |    79.5  | 37.25 0.958 | 39.02 0.967 | 41.05 0.971 | 46.15 0.983 | 35.80 0.969 | 37.08 0.970 | 37.57 0.963 | 35.10 0.966 | 40.02 0.970 | 34.59 0.956 | 38.36 0.967 |
| PADUT [26]         | ICCV 2023   | 5.38M    |    90.46 | 37.36 0.962 | 40.43 0.978 | 42.38 0.979 | 46.62 0.990 | 36.26 0.974 | 37.27 0.974 | 37.83 0.966 | 35.33 0.974 | 40.86 0.978 | 34.55 0.963 | 38.89 0.974 |
| RDLUF [13]         | CVPR 2023   | 1.89M    |   115.34 | 37.94 0.966 | 40.95 0.977 | 43.25 0.979 | 47.83 0.990 | 37.11 0.976 | 37.47 0.975 | 38.58 0.969 | 35.50 0.970 | 41.83 0.978 | 35.23 0.962 | 39.57 0.974 |
| DERNN (9stg) [14]  | TGRS 2024   | 0.65M    |    81.99 | 38.26 0.965 | 40.97 0.979 | 43.22 0.979 | 48.10 0.991 | 38.08 0.980 | 37.41 0.975 | 38.83 0.971 | 36.41 0.973 | 42.87 0.981 | 35.15 0.962 | 39.93 0.976 |
| DERNN (9stg)* [14] | TGRS 2024   | 1.04M    |   134.18 | 38.49 0.968 | 41.27 0.980 | 43.97 0.980 | 48.61 0.992 | 38.29 0.981 | 37.81 0.977 | 39.30 0.973 | 36.51 0.974 | 43.38 0.983 | 35.61 0.966 | 40.33 0.977 |
| SSR-L* [52]        | CVPR 2024   | 1.73M    |    78.93 | 38.81 0.968 | 41.51 0.979 | 43.76 0.979 | 48.62 0.988 | 38.32 0.979 | 37.85 0.975 | 38.50 0.969 | 36.85 0.974 | 42.64 0.980 | 35.82 0.965 | 40.27 0.976 |
| CS-Mamba (3stg)    | Ours        | 0.97M    |    37.8  | 37.93 0.968 | 40.00 0.978 | 42.73 0.983 | 47.18 0.993 | 36.48 0.976 | 37.10 0.976 | 37.91 0.970 | 35.39 0.974 | 41.43 0.981 | 34.82 0.965 | 39.10 0.977 |
| CS-Mamba (5stg)    | Ours        | 0.97M    |    62.93 | 38.23 0.971 | 40.62 0.982 | 43.44 0.984 | 47.58 0.994 | 37.95 0.982 | 37.74 0.979 | 38.71 0.974 | 36.27 0.979 | 42.89 0.986 | 35.45 0.970 | 39.90 0.980 |
| CS-Mamba (9stg)    | Ours        | 0.97M    |   113.18 | 38.51 0.973 | 41.30 0.984 | 43.77 0.985 | 47.55 0.994 | 38.72 0.985 | 38.05 0.981 | 38.88 0.976 | 36.64 0.981 | 43.06 0.987 | 35.54 0.971 | 40.21 0.982 |
| CS-Mamba* (9stg)   | Ours        | 1.16M    |   124.98 | 38.68 0.974 | 41.55 0.985 | 43.99 0.986 | 47.86 0.994 | 38.85 0.986 | 38.15 0.982 | 39.09 0.976 | 36.86 0.983 | 43.25 0.988 | 35.78 0.971 | 40.42 0.983 |
| CS-Mamba-s (9stg)  | Ours        | 0.57M    |    65.77 | 38.63 0.972 | 40.90 0.983 | 43.46 0.984 | 47.30 0.994 | 38.41 0.983 | 37.68 0.979 | 38.80 0.975 | 36.29 0.979 | 43.09 0.986 | 35.41 0.969 | 40.00 0.980 |

In Fig. 5, we display visual results for three Transformerbased methods [2, 3, 5] in a real scene. While their simulation performance improves from left to right, visual quality degrades in real scenarios. Compared to MST, CST's reconstructed image shows more noise artifacts on the nose and forehead. These spots, absent in ground truth, suggest noise from 2D compressed measurements. DAUHST's results exhibit even greater degradation, which is consistently observed across other algorithms in this scene.

This effect stems from two factors. First, simulationbased image reconstruction, aimed at minimizing compression loss, struggles with unknown real-world noise distributions. While shot noise is added during training, it inadequately reflects real-world complexities, leading to insufficient generalization. Secondly, focusing on mask pattern recovery can overemphasize specific information, weakening resilience to other noise types, and creating a balance challenge between compression and real-world losses.

$$T o t a l \ L o s s = C o m p r e s s i o n \ L o s s + \epsilon, \quad ( 1 7 ) \quad \text{bot}$$

where ϵ represents the complex real noise.

We propose a masked training strategy inspired by [8] for SCI reconstruction to enhance robustness and generalization. During training, a randomly generated 0-1 mask is applied to feature maps, creating an inpainting task that compels the network to reconstruct occluded regions. This strengthens the model's capacity to handle complex real-world scenarios. Unlike traditional image denoising, where noise is added to clean images, SCI reconstruction starts with compressed 2D measurements that inherently carry compression loss, compounded by mixed noise in practice. By consistently applying the same mask during both training and testing, we ensure alignment between phases, leading to improved reconstruction accuracy and visually appealing results. This approach addresses unique SCI challenges, enhancing model performance under varied real-world conditions.

## 4. Experiment

## 4.1. Experimental Settings

The effectiveness of our CS-Mamba network is verified on both simulation and real datasets, by using 28 wavelengths ranging from 450nm to 650nm, acquired via spectral interpolation manipulation.

Figure 6. Visual comparison of different methods on simulation Scene 2 at wavelength 575.5nm

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-bb7d6ecd-f0ce-410e-ac34-6ed42e2b2d89.jpg)

Simulation HSI Data. Two simulated HSI datasets CAVE [38] and KAIST [10] are utilized to conduct the simulation experiment. The CAVE dataset contains 32 HSIs with a spatial size of 512 × 512 , while the KAIST dataset includes 30 HSIs with a spatial size of 2704 × 3376 . Following the experimental settings of TSA-Net [32], we designate CAVE as the training set and select 10 scenes from the KAIST as the testing set.

Real HSI Data. We adopt the real HSIs obtained by the CASSI system, as developed in TSA-Net [32] for testing. Evaluation Metrics. We select peak signal-to-noise ratio (PSNR) [7] and structural similarity (SSIM) [46] as the evaluation metrics to assess the quality of HSI reconstruction.

Implementation Details. Our CS-Mamba is implemented using PyTorch and we conduct model training with Adam optimizer ( β 1 = 0.9 and β 2 = 0.999) for 300 epochs. The learning rate is set to 1 × 10 -3 at the beginning and a cosine annealing scheduler is adopted. Image patches with size 256 × 256 are randomly cropped from the 3D HSI cubes with 28 channels for training in the simulation and real experiments. The shifting step d in the dispersion process is set to 2. We apply the data augmentation techniques including random flipping and rotation. Models are trained on RTX 4090 GPUs.

## 4.2. Simulation Results

Weconduct a comprehensive comparative analysis of our CSMamba method and SOTA HSI reconstruction algorithms in Params, FLOPs, PSNR, and SSIM. We compared the results of CS-Mamba with 11 SOTA methods algorithms across 10 simulation scenes. The quantitative results of different models are listed in Table 3.

As shown in Table 3, our CS-Mamba-9stg* obtains 40.42dB of PSNR and 0.983 of SSIM, achieving the best reconstruction performance among SOTAs. When compared to recent Transformer-based methods such as DAUHST [5], RDLUF [13], DERNN [14], and SSR-L* [52], our CS-Mamba demonstrates notable improvements over these methods, which are 2.06dB, 0.85dB, 0.49dB, and 0.15dB on average, respectively.

We also conduct comparisons of our CS-Mamba-S with previous Transformer-based methods DAUHST [5] and

DERNN [14]. As shown in Table 2, we not only achieve better performance in PSNR and SSIM compared to DAUHST and DERNN but also obtain a more lightweight model. Specifically, the CS-Mamba has 0.08 (M) fewer parameters than the DERNN and 13.73 (G) fewer in FLOPs, which means the smallest parameter count and Computational cost among recent deep unfolding methods. Furthermore, we evaluated the denoiser network (Cross-Scan SSM Block) as an independent end-to-end model, comparing it against previous end-to-end Transformer methods, as presented in Table 1. Specifically, the Spatial-Spectral SSM is 2.4 times faster than MST-S and 4.5 times faster than CST-S, while having 0.12 (M) fewer parameters than the MST-S, which means the smallest parameter count.

Except for quantitative comparisons, we also visualize the results of simulation experiments. We choose one of 28 channels from Scene 2, against the simulation results obtained from eleven SOTA approaches. Visually, as illustrated in Fig. 6, our method yields smoother and cleaner details. The experimental results indicate the capability of our method to generate high-quality HSIs.

## 4.3. Real Data Results

We apply our model and the proposed masked strategy to reconstruct real HSIs, retraining on the CA VE [38] and KAIST [10] datasets and simulating real conditions with 11-bit shot noise. Fig. 7 compares reconstructions from two real scenes using our original CS-Mamba model, the masked version, and four state-of-the-art methods. When enlarging the image results, we can clearly observe that the original CS-Mamba output exhibits a peculiar grid-like noise around the shoulders and face of the subject. In contrast, our mask method effectively mitigates this issue, producing a more visually pleasing effect compared to previous approaches, achieving smoother and more accurate reconstruction.

We further conducted a separate comparison between the original model and the model employing the masked strategy in Scene 1. As shown in Fig. 4, the original model exhibits some wrinkles and artifacts on the raised areas of the petals, with edges becoming blurred, and the stem displaying certain noise patterns. In contrast, the ground truth image presents smooth elevations. This issue is significantly alleviated with

Ground Truth

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00629v2-figure-b7284c86-6a2b-4c1b-8c8d-e596958c9a5c.jpg)

GAP-Net

DGSMP

DiffSCI

DAUHST

Ours (3stg)

Ours (Mask)

Figure 7. Visual comparison of SCI reconstruction methods on real Scene 3 at wavelength 648.0nm.

our masked method, obtaining a marked improvement in visual quality. Additional experiments will be included in the supplementary material to further substantiate the validity of our approach and conclusions, while Visual comparisons here sufficiently demonstrate the effectiveness of our masked training method.

## 4.4. Ablation Study

Break-down Ablation . We conduct an ablation study by removing the LE-SSM block and CS-SSM block from the CS-Mamba net-

Table 4. Ablation study of CS-Mamba (3stg)'s components

under both approaches in our CS-Mamba (3stg) network. As shown in Table 5, common scanning provided a 0.12 dB improvement, whereas the proposed across-scanning mechanism yielded a significant 0.27 dB increase. Both methods were tested under single-scan conditions, with our method excelling due to its enhanced focus on adjacent bands and pixels, effectively capturing local features across spatial and spectral dimensions. These results underscore the effectiveness of our across-scanning approach.

| Architecture            |   Params (M) PSNR |   Params (M) PSNR | SSIM        |
|-------------------------|-------------------|-------------------|-------------|
| Baseline w/o SSM        |              0.37 |             37.41 | 0.964       |
| with LE-SSM with CS-SSM |              0.77 |             38.83 | 0.975 0.968 |
|                         |              0.57 |             37.88 |             |
| with LE + CS SSM        |              0.97 |             39.1  | 0.977       |

work. The experiment results are shown in Table 4. Our goal is to explore the contributions of each mechanism to the results. The baseline barely adopts the parameter estimation, U-shaped framework, and GDFFN block without any SSM modules. Since [14] has demonstrated the effectiveness of GDFFN block, we don't conduct an ablation study for it.

When we respectively employ LE-SSM and CS-SSM blocks, our model achieves 1.42 dB and 0.47 dB improvements in PSNR over the baseline, respectively. Employing both mechanisms simultaneously can further enhance the model's performance. Here we need to point out that, in consideration of maintaining lightweight, we only utilized a single-direction scan in CS-SSM. In fact, we could also deploy a four-direction scanning similar to LE-SSM. However, each scan direction implies additional computational costs so we decided not to adopt this way.

## Cross-Scan Mechanism .

To highlight the advantages of our cross-scan mechanism over common spectral scanning, we compared model performance

Table 5. Comparisons of CS mechanism and common scanning.

| Architecture             |   PSNR | SSIM        |
|--------------------------|--------|-------------|
| w/o Spectral SSM         |  38.83 | 0.975       |
| with CS Mechanism Common |  39.1  | 0.977 0.975 |
| with Scanning            |  38.95 |             |

## Local-Enhancement

Mechanism . We also conducted an ablation study of our LE-SSM mechanism in our CSMamba (3stg) to assess its impact. As shown in

Table 6. Comparisons of LE mechanism and global scanning.

| Architecture                      | PSNR SSIM               |
|-----------------------------------|-------------------------|
| w/o Spatial SSM                   | 37.88 0.968             |
| with LE Machenism Global Scanning | 38.83 0.975 38.66 0.974 |
| with                              |                         |

Table 6, starting from a baseline model with a PSNR of 37.88 dB (without spatial SSM), incorporating global four-direction scanning improved PSNR by 0.95 dB and SSIM by 0.007, demonstrating effective global spatial context modeling. Applying our LE mechanism along the spatial dimension further improved PSNR by 0.17 dB compared to global scanning, without introducing additional parameters or computational costs, unlike methods using CNN layers or spatial attention [24]. This enhancement highlights the significance of capturing local features in HSIs and validates the effectiveness of our LE mechanism.

## 5. Conclusion

In this paper, we propose a novel cross-scan SSM mechanism that more fully leverages the spectral and spatial local features of HSI images, achieving improved performance compared to the common SSM mechanism. Additionally, we identify and elaborate on noise issues encountered in real-world scene reconstruction, presenting a masked training method designed to address these challenges. This ap-

proach significantly mitigates the degradation problems in real-scene reconstruction that previous methods failed to resolve, enhancing the robustness of the model. Experiments in the main text and supplementary material validate the effectiveness of our method.

## References

- [1] V Backman, Michael B Wallace, LT Perelman, JT Arendt, R Gurjar, MG Müller, Q Zhang, G Zonios, E Kline, T McGillican, et al. Detection of preinvasive cancer cells. Nature , 406 (6791):35-36, 2000. 1
- [2] Yuanhao Cai, Jing Lin, Xiaowan Hu, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Luc Van Gool. Coarseto-fine sparse transformer for hyperspectral image reconstruction. In Proceedings of the European Conference on Computer Vision , pages 686-704. Springer, 2022. 2, 4, 5, 6
- [3] Yuanhao Cai, Jing Lin, Xiaowan Hu, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Luc Van Gool. Maskguided spectral-wise transformer for efficient hyperspectral image reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17502-17511, 2022. 1, 4, 5, 6
- [4] Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Radu Timofte, and Luc Van Gool. Mst++: Multi-stage spectral-wise transformer for efficient spectral reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 745-755, 2022.
- [5] Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Henghui Ding, Yulun Zhang, Radu Timofte, and Luc V Gool. Degradation-aware unfolding half-shuffle transformer for spectral compressive imaging. In Proceedings of the Advances in Neural Information Processing Systems , pages 37749-37761, 2022. 2, 4, 5, 6, 7
- [6] Xun Cao, Tao Yue, Xing Lin, Stephen Lin, Xin Yuan, Qionghai Dai, Lawrence Carin, and David J Brady. Computational snapshot multispectral cameras: Toward dynamic capture of the spectral world. IEEE Signal Processing Magazine , 33(5): 95-108, 2016. 1
- [7] Luen C. Chan and Peter Whiteman. Hardware-constrained hybrid coding of video imagery. IEEE Transactions on Aerospace and Electronic Systems , AES-19(1):71-84, 1983. 7
- [8] Haoyu Chen, Jinjin Gu, Yihao Liu, Salma Abdel Magid, Chao Dong, Qiong Wang, Hanspeter Pfister, and Lei Zhu. Masked image training for generalizable deep image denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1692-1703, 2023. 6
- [9] Yongyong Chen, Yanwen Guo, Yongli Wang, Dong Wang, Chong Peng, and Guoping He. Denoising of hyperspectral images using nonconvex low rank matrix approximation. IEEE Transactions on Geoscience and Remote Sensing , 55(9):53665380, 2017. 1
- [10] Inchang Choi, MH Kim, D Gutierrez, DS Jeon, and G Nam. High-quality hyperspectral reconstruction using a spectral prior. In Technical report , 2017. 7
- [11] Gu T. Deng R. Cu-mamba: Selective state space models with channel learning for image restoration. arXiv preprint arXiv:2404.11778 , 2024. 3, 5
- [12] Weisheng Dong, Chen Zhou, Fangfang Wu, Jinjian Wu, Guangming Shi, and Xin Li. Model-guided deep hyperspectral image super-resolution. IEEE Transactions on Image Processing , 30:5754-5768, 2021. 1
- [13] Yubo Dong, Dahua Gao, Tian Qiu, Yuyan Li, Minxi Yang, and Guangming Shi. Residual degradation learning unfolding framework with mixing priors across spectral and spatial for compressive spectral imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 22262-22271, 2023. 6, 7
- [14] Yubo Dong, Dahua Gao, Yuyan Li, Guangming Shi, and Danhua Liu. Degradation estimation recurrent neural network with local and non-local priors for compressive spectral imaging. IEEE Transactions on Geoscience and Remote Sensing , 62:1-15, 2024. 4, 5, 6, 7, 8
- [15] Hao Du, Xin Tong, Xun Cao, and Stephen Lin. A prism-based system for multispectral video acquisition. In 2009 IEEE 12th International Conference on Computer Vision , pages 175-182. IEEE, 2009. 1
- [16] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 , 2023. 2
- [17] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 , 2023. 3
- [18] Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In ICLR , 2021. 3
- [19] Hang Guo, Jinmin Li, Tao Dai, Zhihao Ouyang, Xudong Ren, and Shu-Tao Xia. Mambair: A simple baseline for image restoration with state-space model. arXiv preprint arXiv:2402.15648 , 2024. 2
- [20] Xian-Hua Han and Yen-Wei Chen. Deep residual network of spectral and spatial fusion for hyperspectral image superresolution. In 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM) , pages 266-270. IEEE, 2019. 1
- [21] Wei He, Quanming Yao, Chao Li, Naoto Yokoya, Qibin Zhao, Hongyan Zhang, and Liangpei Zhang. Non-local meets global: An iterative paradigm for hyperspectral image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(4):2089-2107, 2020. 1
- [22] Lingbo Huang, Yushi Chen, and Xin He. Spectral-spatial mamba for hyperspectral image classification. arXiv preprint arXiv:2404.18401 , 2024. 2
- [23] Tao Huang, Weisheng Dong, Xin Yuan, Jinjian Wu, and Guangming Shi. Deep gaussian scale mixture prior for spectral compressive imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16216-16225, 2021. 6
- [24] Tao Huang, Xiaohuan Pei, Shan You, Fei Wang, Chen Qian, and Chang Xu. Localmamba: Visual state space model with windowed selective scan. arXiv preprint arXiv:2403.09338 , 2024. 5, 8

| [25]   | M. H. Kim, T. A. Harvey, D. S. Kittle, H. Rushmeier, R. O. Prum J. Dorsey, and D. J. Brady. 3d imaging spec- troscopy for measuring hyperspectral patterns on solid objects. ACM Transactions on on Graphics , 2012. 1                                        |
|--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [26]   | Miaoyu Li, Ying Fu, Ji Liu, and Yulun Zhang. Pixel adaptive deep unfolding transformer for hyperspectral image recon- struction. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 12959-12968, 2023. 6                      |
| [27]   | Weibin Liao, Yinghao Zhu, Xinyuan Wang, Cehngwei Pan, Yasha Wang, and Liantao Ma. Lightm-unet: Mamba assists in lightweight unet for medical image segmentation. arXiv preprint arXiv:2403.05246 , 2024. 2                                                    |
| [28]   | Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. arXiv preprint arXiv:2401.10166 , 2024. 3, 5                                                                          |
| [29]   | Patrick Llull, Xuejun Liao, Xin Yuan, Jianbo Yang, David Kittle, Lawrence Carin, Guillermo Sapiro, and David J Brady. Coded aperture compressive temporal imaging. Optics Ex- press , 2013. 1                                                                 |
| [30]   | Guolan Lu and Baowei Fei. Medical hyperspectral imaging: a review. Journal of Biomedical Optics , 2014. 1                                                                                                                                                     |
| [31]   | Jiawei Ma, Xiao-Yang Liu, Zheng Shou, and Xin Yuan. Deep tensor admm-net for snapshot compressive imaging. In Pro- ceedings of the IEEE/CVF International Conference on Com- puter Vision , pages 10223-10232, 2019. 6                                        |
| [32]   | Ziyi Meng, Jiawei Ma, and Xin Yuan. End-to-end low cost compressive spectral imaging with spatial-spectral self- attention. In Proceedings of the European Conference on Computer Vision , pages 187-204. Springer, 2020. 2, 7                                |
| [33]   | Ziyi Meng, Mu Qiao, Jiawei Ma, Zhenming Yu, Kun Xu, and Xin Yuan. Snapshot multispectral endomicroscopy. Optics Letters , 2020. 1                                                                                                                             |
| [34]   | Ziyi Meng, Zhenming Yu, Kun Xu, and Xin Yuan. Self- supervised neural networks for spectral snapshot compressive imaging. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2622-2631, 2021. 6                               |
| [35]   | Xin Miao, Xin Yuan, Yunchen Pu, and Vassilis Athitsos. λ - net: Reconstruct hyperspectral images from a snapshot mea- surement. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4059-4069, 2019. 2                         |
| [36]   | Zhihong Pan, Glenn Healey, Manish Prasad, and Bruce Tromberg. Face recognition in hyperspectral images. IEEE Transactions on Pattern Analysis and Machine Intelligence , 25(12):1552-1560, 2003. 1                                                            |
| [37]   | Zhenghao Pan, Haijin Zeng, Jiezhang Cao, Kai Zhang, and Yongyong Chen. Diffsci: Zero-shot snapshot compressive imaging via iterative spectral diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024. 6 |
| [38]   | Jong-Il Park, Moon-Hyun Lee, Michael D Grossberg, and Shree K Nayar. Multispectral imaging using multiplexed illumination. In 2007 IEEE 11th International Conference on Computer Vision , pages 1-8. IEEE, 2007. 7                                           |
| [39]   | Yuan Shi, Bin Xia, Xiaoyu Jin, Xing Wang, Tianyu Zhao, Xin Xia, Xuefeng Xiao, and Wenming Yang. Vmambair:                                                                                                                                                     |

- Visual state space model for image restoration. arXiv preprint arXiv:2403.11423 , 2024. 2
- [40] Hien Van Nguyen, Amit Banerjee, and Rama Chellappa. Tracking via object reflectance using a hyperspectral video camera. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops , pages 4451. IEEE, 2010. 1
- [41] Ashwin Wagadarikar, Renu John, Rebecca Willett, and David Brady. Single disperser design for coded aperture snapshot spectral imaging. Applied Optics , 2008. 1
- [42] Ashwin A Wagadarikar, Nikos P Pitsianis, Xiaobai Sun, and David J Brady. Video rate spectral imaging using a coded aperture snapshot spectral imager. Optics Express , 2009. 1
- [43] Guanchun Wang, Xiangrong Zhang, Zelin Peng, Tianyang Zhang, Xiuping Jia, and Licheng Jiao. S2mamba: A spatialspectral state space model for hyperspectral image classification. arXiv preprint arXiv:2404.18213 , 2024. 2, 3, 5
- [44] Hongyuan Wang, Lizhi Wang, Chang Chen, Xue Hu, Fenglong Song, and Hua Huang. Learning spectral-wise correlation for spectral super-resolution: Where similarity meets particularity. In Proceedings of the 31st ACM International Conference on Multimedia , pages 7676-7685, 2023. 1
- [45] Xiao Wang, Shiao Wang, Yuhe Ding, Yuehang Li, Wentao Wu, Yao Rong, Weizhe Kong, Ju Huang, Shihao Li, Haoxiang Yang, et al. State space model for new-generation network alternative to transformers: A survey. arXiv preprint arXiv:2404.09516 , 2024. 2
- [46] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing , 13(4):600-612, 2004. 7
- [47] Shumian Yang, Xinxin Xiang, Fenghua Tong, Dawei Zhao, and Xin Li. Image compressed sensing using multi-scale characteristic residual learning. In 2023 IEEE International Conference on Multimedia and Expo (ICME) , pages 15951600. IEEE, 2023. 1
- [48] Yangke Ying, Jin Wang, Yunhui Shi, and Baocai Yin. Dualdomain feature learning and memory-enhanced unfolding network for spectral compressive imaging. In 2023 IEEE International Conference on Multimedia and Expo (ICME) , pages 1589-1594. IEEE, 2023. 1
- [49] Qiangqiang Yuan, Qiang Zhang, Jie Li, Huanfeng Shen, and Liangpei Zhang. Hyperspectral image denoising employing a spatial-spectral deep residual convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing , 57 (2):1205-1218, 2018. 1
- [50] Xin Yuan. Generalized alternating projection based total variation minimization for compressive sensing. In 2016 IEEE International conference on image processing (ICIP) , pages 2539-2543. IEEE, 2016. 1
- [51] Xin Yuan, Tsung-Han Tsai, Ruoyu Zhu, Patrick Llull, David Brady, and Lawrence Carin. Compressive hyperspectral imaging with side information. IEEE Journal of selected topics in Signal Processing , 9(6):964-976, 2015. 1
- [52] Jiancheng Zhang, Haijin Zeng, Yongyong Chen, Dengxiu Yu, and Yin-Ping Zhao. Improving spectral snapshot reconstruction with spectral-spatial rectification. In Proceedings of

the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 25817-25826, 2024. 6, 7

- [53] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417 , 2024. 2, 3
- [54] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417 , 2024. 2
Wenzhe Tian Haijin Zeng Yin-Ping Zhao Yongyong Chen Zhen Wang Xuelong Li
Cross-Scan Mamba with Masked Training for Robust Spectral Imaging
Snapshot Compressive Imaging (SCI) enables fast spectral imaging but requires
effective decoding algorithms for hyperspectral image (HSI) reconstruction from
compressed measurements. Current CNN-based methods are limited in modeling
long-range dependencies, while Transformer-based models face high computational
complexity. Although recent Mamba models outperform CNNs and Transformers in
RGB tasks concerning computational efficiency or accuracy, they are not
specifically optimized to fully leverage the local spatial and spectral
correlations inherent in HSIs. To address this, we propose the Cross-Scanning
Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local
balanced context encoding and cross-channel interaction promotion. Besides,
while current reconstruction algorithms perform increasingly well in simulation
scenarios, they exhibit suboptimal performance on real data due to limited
generalization capability. During the training process, the model may not
capture the inherent features of the images but rather learn the parameters to
mitigate specific noise and loss, which may lead to a decline in reconstruction
quality when faced with real scenes. To overcome this challenge, we propose a
masked training method to enhance the generalization ability of models.
Experiment results show that our CS-Mamba achieves state-of-the-art performance
and the masked training method can better reconstruct smooth features to
improve the visual quality.