## PhysMamba: Synergistic State Space Duality Model for Remote Physiological Measurement

Zhixin Yan 1 , Yan Zhong 1 , Hongbin Xu 1 , Wenjun Zhang 1 , Shangru Yi 1 , Lin Shu 1 , Wenxiong Kang 1 ∗ South China University of Technology, Guangzhou, China

Abstract -Remote Photoplethysmography (rPPG) enables noncontact physiological signal extraction from facial videos, offering applications in psychological state analysis, medical assistance, and anti-face spoofing. However, challenges such as motion artifacts, lighting variations, and noise limit its real-world applicability. To address these issues, we propose PhysMamba, a novel dual-pathway time-frequency interaction model based on Synergistic State Space Duality (SSSD), which for the first time integrates state space models with attention mechanisms in a dual-branch framework. Combined with a Multi-Scale Query (MQ) mechanism, PhysMamba achieves efficient information exchange and enhanced feature representation, ensuring robustness under noisy and dynamic conditions. Experiments on PURE, UBFC-rPPG, and MMPD datasets demonstrate that PhysMamba outperforms state-of-the-art methods, offering superior accuracy and generalization. This work lays a strong foundation for practical applications in non-contact health monitoring, including real-time remote patient care. The code is available at https://anonymous.4open.science/r/PhysMamba-E714/.

Index Terms -Remote Photoplethysmography, Heart Rate Estimation, Mamba, State Space Duality

## I. INTRODUCTION

Photoplethysmography (PPG) is a widely-used technique for estimating vital physiological indicators such as heart rate, heart rate variability, and respiratory rate, which are crucial for applications like sleep monitoring, medical assistance, and fatigue assessment. While traditional PPG relies on contactbased sensors like ECG monitors, its limited scalability and user inconvenience have driven the development of noncontact alternatives. Remote photoplethysmography (rPPG) leverages cameras to estimate physiological signals by capturing subtle blood flow changes in the skin [1], unlocking potential applications in remote health monitoring, wearable devices, and psychological state analysis.

Despite its promise, deploying rPPG in real-world environments faces significant challenges, including motion artifacts, lighting variations, and environmental noise. These factors disrupt signal consistency and hinder the generalization of rPPG systems, limiting their practical adoption in scenarios such as telemedicine and stress monitoring.

Traditional rPPG methods, including signal processing techniques like Independent Component Analysis (ICA) and chrominance-based projections [2], [3], perform well in controlled environments but struggle under dynamic conditions due to their reliance on handcrafted features. Recent deep learning approaches, such as CNN-based models [23], have improved motion artifact handling, while Transformer-based

Fig. 1. Comparison between SSD and SSSD. The traditional SSD combines SSM and attention mechanisms, while our enhanced SSSD introduces MultiScale Queries for efficient information interaction between the two pathways.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01077v3-figure-627d2ade-1616-4b17-8492-d38202aba097.jpg)

architectures like PhysFormer [29] enhance temporal dependency modeling. However, these models suffer from high computational complexity, making them less practical for realtime applications. State Space Models (SSMs) have emerged as a computationally efficient alternative, as demonstrated by RhythmMamba [10], which captures quasi-periodic patterns in physiological signals with low computational cost. Yet, existing SSM-based methods face limitations in generalization and parallel processing, particularly in cross-dataset scenarios.

Building on these advancements, we propose PhysMamba , a novel dual-pathway time-frequency interaction model that integrates Synergistic State Space Duality (SSSD) for rPPG estimation. This is the first framework to combine the efficiency of state space models with the feature extraction capabilities of attention mechanisms in a dual-pathway architecture. By introducing a Multi-Scale Query (MQ) mechanism, PhysMamba achieves efficient information sharing between pathways, enabling robust feature representation under noisy and dynamic conditions.

Comprehensive experiments on the PURE, UBFC-rPPG, and MMPD datasets demonstrate that PhysMamba achieves state-of-the-art performance in both intra-dataset and crossdataset evaluations. These results highlight its superior robustness, accuracy, and generalization, paving the way for practical applications in non-contact health monitoring systems, including wearable devices and real-time telemedicine.

The main contributions of this work are summarized as follows:

- · We propose PhysMamba, an efficient rPPG estimation method via State Space Duality in a dual-pathway time-

frequency interactive network. This is the first attempt handling rPPG estimation via SSD, creating technological breakthroughs.

- · To enhance information exchange in dual-pathway model, we develop a simple state space fusion module using Mutil-Scale Queries for effective lateral information sharing.
- · Extensive experiments on three public rPPG datasets (PURE, UBFC-rPPG, and MMPD) validate the superior performance of PhysMamba, achieving state-of-the-art results in both controlled and complex real-world scenarios.

## II. RELATED WORK

## A. rPPG Estimation

Traditional rPPG methods, such as Independent Component Analysis (ICA) and chrominance subspace projection [2], [21], perform well under controlled conditions but degrade significantly in dynamic environments due to sensitivity to motion artifacts and lighting variations [25]. These approaches also rely heavily on handcrafted features, limiting their adaptability in real-world scenarios. Deep learning has advanced rPPG estimation by enabling end-to-end frameworks. Models like DeepPhys [23] improved motion artifact handling through CNN architectures but struggled to capture long-term dependencies. Transformer-based methods, such as PhysFormer [29], leveraged self-attention mechanisms for global dependency modeling but suffered from high computational complexity, making them impractical for real-time applications.

## B. State Space Model

To enhance contextual information capture in long sequences, SSMs have gained significant interest. The Mamba SSM [6] maintains linear time complexity while effectively capturing long-term dependencies, and has been applied in fields such as time series analysis [7], medical image segmentation [8], and video understanding [9]. Recently, Mamba has been used for rPPG estimation, achieving good results with low computational complexity. RhythmMamba [10], an SSM-based method, robustly identifies quasi-periodic patterns in rPPG signals using a multi-phase learning framework and a frequency-domain feedforward mechanism. However, Mamba1 had limitations in efficiency and generalization due to its dependency on previous states. Mamba-2 [11] introduced the State Space Duality framework, enhancing SSMs' linear time complexity with attention mechanisms' feature representation capabilities, improving efficiency and generalization for complex physiological signal estimation tasks like rPPG.

## III. METHODOLOGY

## A. Overall Framework

PhysMamba is an end-to-end model designed to process raw facial video data and estimate rPPG signals directly. The framework, illustrated in Figure 2, consists of two pathways: the Self-Attention (SA) Pathway and the Cross-Attention (CA) Pathway. These pathways incorporate three core modules: the

Frame Stem for temporal feature extraction, the Multi-Scale Synergistic State Space Duality (SSSD) module for robust temporal representation, and the Frequency Domain FeedForward (FDF) network for frequency enhancement. Outputs from these pathways are fused in the rPPG Predictor to generate final signal estimations. The dual-pathway architecture can be likened to a collaborative system where one pathway focuses on internal temporal consistency (SA) and the other facilitates cross-feature interactions (CA), ensuring robustness in diverse conditions.

## B. Frame Stem

The Frame Stem extracts temporal features while reducing spatial noise from input video data X ∈ R 3 × × T H × W . By computing frame differences, subtle temporal variations are highlighted, which are particularly relevant for physiological signal estimation. The processed features are represented as:

$$X _ { \text{Stem} } = \text{Stem} ( X _ { \text{origin} } ) + \text{Stem} ( X _ { \text{origin} } + X _ { \text{diff} } ), \quad ( 1 )$$

where X diff represents differences between consecutive frames.

A combination of 2D-CNNs, batch normalization, ReLU activation, and max-pooling is used to enhance these features. Finally, a 5×5 convolution and self-attention mechanism adaptively focus on informative facial regions, improving feature quality for subsequent modules.

## C. Multi-Scale Synergistic State Space Duality (SSSD)

The SSSD module combines State Space Models (SSMs) with attention mechanisms to capture both long-term dependencies and short-term variations in temporal data between the two pathways, as illustrated in Fig. 2(a). A key component of SSSD is the Multi-Scale Query (MQ) mechanism, which allows the model to process information across multiple temporal scales, as shown in Fig. 2(b). By ensuring efficient information exchange between the pathways, MQ enhances the model's adaptability to diverse and dynamic input characteristics, such as motion artifacts and inconsistent lighting. This design allows PhysMamba to excel in both time-domain and frequency-domain tasks, ensuring high performance in realworld scenarios.

To simplify, SSMs operate like 'memory systems' that track evolving patterns over time. They are mathematically described as:

$$h _ { t } = A h _ { t - 1 } + B x _ { t }, \ \ y _ { t } = C ^ { \top } h _ { t }, \ \quad \ \ ( 2 )$$

where A B , , and C are learnable matrices controlling state transitions and feature outputs.

Building on this, the State Space Duality (SSD) framework introduces structured attention mechanisms to efficiently encode temporal dependencies:

$$\text{SSD} = ( L \circ Q K ^ { \top } ) \cdot V,$$

Through the MQ mechanism, the query ( Q ), key ( K ), and value ( V ) matrices are mapped from inputs of different temporal scales, with L as a structured mask matrix that replaces traditional softmax operations.

## (a) The overall framework diagram of PhysMamba

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01077v3-figure-3a64405c-f4c0-4efc-ad66-4553e4c6b349.jpg)

Fig. 2. Overall framework diagram of the PhysMamba model. The model comprises the Frame Stem, Multi-Scale Synergistic State Space Duality (SSSD), Frequency Domain Feed-Forward (FDF), and rPPG Predictor, integrated into two pathways: the Self-Attention Pathway and the Cross-Attention Pathway.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01077v3-figure-b53d74a3-067a-4226-bfdf-cf89ba288042.jpg)

The SSSD extends this concept by enabling complementary interactions between pathways:

from the SA and CA pathways, denoted as F SA and F CA , are combined to optimize rPPG signal representation.

The SA Pathway focuses on self-dependencies within individual features:

$$X _ { \text{SA} } = ( L _ { S } \circ Q _ { S } K _ { S } ^ { \top } ) \cdot V _ { S }. \quad \quad ( 4 ) \ \text{ along}$$

The CA pathway receives Multi-Scale Q S from SA, enabling CA to promote cross-feature refinement.:

$$X _ { \text{CA} } = ( L _ { C } \circ Q _ { S } K _ { C } ^ { \top } ) \cdot V _ { C }. \quad \quad ( 5 ) \prod _ { \substack { a \dots \dots } }$$

As illustrated in Fig. 2(c), these pathways function like two analysts: one focuses on deeply examining individual data streams (SA), while the other identifies meaningful interactions across them (CA). This collaboration ensures robust temporal feature learning, even under noisy conditions.

## D. Frequency Domain Feed-Forward (FDF)

To enhance periodic physiological patterns, the FDF network applies Fast Fourier Transform (FFT) to convert temporal signals h t ( ) into the frequency domain:

$$H ( f ) = \int _ { - \infty } ^ { + \infty } h ( t ) e ^ { - j 2 \pi f t } d t. \quad \quad ( 6 ) \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{  \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{} \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{$$

This emphasizes periodic components like heart rate signals. The inverse FFT restores the frequency-enhanced signals to the temporal domain, preserving these enhancements. Outputs

## E. rPPG Predictor

The outputs from the SA and CA pathways are concatenated along the channel dimension:

$$X _ { \text{fusion} } = \text{Concat} ( F _ { C A }, F _ { S A } ). \text{ \quad \ \ } ( 7 )$$

A 1D convolutional layer then refines the fused features to produce the final rPPG predictions. By integrating both timedomain and frequency-domain features, the rPPG Predictor ensures accurate heart rate estimation even under challenging conditions, such as varying lighting and motion artifacts.

## IV. EXPERIMENTS

## A. Datasets

To rigorously evaluate PhysMamba's performance, we selected three widely-used rPPG datasets with varying levels of complexity: UBFC-rPPG [32], PURE [31], and MMPD [33]. These datasets present distinct challenges, ranging from controlled laboratory conditions to highly complex real-world scenarios involving significant noise and motion artifacts.

UBFC-rPPG : This dataset consists of videos from 42 subjects performing mathematical tasks to induce heart rate variability. The relatively clean facial videos with minimal noise make it a benchmark for assessing model accuracy under simple conditions.

TABLE I INTRA-DATASET RESULTS ON UBFC, PURE, AND MMPD DATASETS.

| Method             | UBFC   | UBFC   | UBFC   | UBFC   | PURE   | PURE   | PURE   | PURE   | MMPD   | MMPD   | MMPD   | MMPD   | MMPD   |
|--------------------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|
| Method             | MAE    | RMSE   | MAPE   | r      | MAE    | RMSE   | MAPE   | r      | MAE    | RMSE   | MAPE   | r      | SNR    |
| GREEN [16]         | 19.73  | 31.00  | 18.72  | 0.37   | 10.09  | 23.85  | 10.28  | 0.34   | 21.68  | 27.69  | 24.39  | -0.01  | -14.34 |
| ICA [18]           | 16.00  | 25.65  | 15.35  | 0.44   | 4.77   | 16.07  | 4.47   | 0.72   | 18.60  | 24.30  | 20.88  | 0.01   | -13.84 |
| CMROM [17]         | 4.06   | 8.83   | 3.34   | 0.89   | 5.77   | 14.93  | 11.52  | 0.81   | 13.66  | 18.76  | 16.00  | 0.08   | -11.74 |
| LGI [19]           | 15.80  | 28.55  | 14.70  | 0.36   | 4.61   | 15.38  | 4.96   | 0.77   | 17.08  | 23.32  | 18.98  | 0.04   | -13.15 |
| PBV [20]           | 15.90  | 26.40  | 15.17  | 0.48   | 3.92   | 12.99  | 4.84   | 0.84   | 17.95  | 23.58  | 20.18  | 0.09   | -13.88 |
| POS [21]           | 4.08   | 7.72   | 3.93   | 0.92   | 3.67   | 11.82  | 7.25   | 0.88   | 12.36  | 17.71  | 14.43  | 0.18   | -11.53 |
| OMIT [22]          | 15.79  | 28.54  | 14.69  | 0.36   | 4.65   | 15.81  | 4.96   | 0.75   | 7.80   | 12.00  | 10.55  | 0.13   | -8.68  |
| DeepPhys [23]      | 0.76   | 1.09   | 0.79   | 0.99   | 3.33   | 14.45  | 2.91   | 0.90   | 23.73  | 28.25  | 25.63  | -0.06  | -15.45 |
| PhysNet [24]       | 0.58   | 0.83   | 0.61   | 0.99   | 0.54   | 0.93   | 0.58   | 0.99   | 4.81   | 11.83  | 4.84   | 0.60   | 1.51   |
| TS-CAN [26]        | 0.81   | 1.10   | 0.84   | 0.99   | 0.40   | 0.73   | 0.44   | 0.99   | 8.97   | 16.58  | 9.43   | 0.44   | -6.92  |
| PhysFormer [29]    | 0.63   | 0.98   | 0.65   | 0.99   | 0.25   | 0.37   | 0.34   | 0.99   | 13.64  | 19.39  | 14.42  | 0.15   | -11.02 |
| EfficientPhys [30] | 0.72   | 1.01   | 0.75   | 0.99   | 5.10   | 16.61  | 4.19   | 0.87   | 12.79  | 21.12  | 13.48  | 0.24   | -9.23  |
| RhythmMamba [10]   | 0.54   | 0.79   | 0.54   | 0.99   | 0.29   | 0.39   | 0.36   | 0.99   | 3.16   | 7.27   | 3.37   | 0.84   | 4.74   |
| PhysMamba (Ours)   | 0.45   | 0.76   | 0.45   | 0.99   | 0.24   | 0.36   | 0.31   | 0.99   | 2.84   | 6.41   | 3.04   | 0.88   | 5.20   |

Fig. 3. Comparison of predicted and ground-truth heart rate distributions across UBFC-rPPG, PURE, and MMPD datasets.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01077v3-figure-b08e781d-5b15-45e1-bfdf-74c3d35c2a07.jpg)

PURE : Comprising videos of 10 subjects under six controlled head movement scenarios-including stillness, talking, and slow/fast rotations-PURE introduces moderate motion artifacts, testing the model's robustness to dynamic conditions.

MMPD : A highly challenging dataset featuring 33 subjects under varying lighting conditions, skin tones, and head movement complexities. Capturing over 11 hours of video with diverse environmental factors, MMPD includes significant noise and motion artifacts, simulating real-world complexities. Due to computational constraints, we utilized the mini-MMPD version for experimentation.

We evaluated PhysMamba using standard metrics: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), and Pearson correlation coefficient ( r ). For MMPD, we also computed the Signal-to-Noise Ratio (SNR) to assess robustness in high-noise environments.

## B. Experimental Details

All experiments were conducted using the open-source rppg-toolbox [34] in PyTorch. Videos were preprocessed by cropping and resizing facial regions to 128 × 128 pixels. The model was trained using a learning rate of 3 × 10 -4 , a batch size of 16, and for 30 epochs. The SSD model's dimension and state space size were set to 64, with a head dimension of 16. All computations were performed on an NVIDIA RTX 4090 GPU.

Fig. 4. Visualization of the results from the MMPD dataset. (a) the BlandAltman plot. (b) the waveform of a sample before and after filtering.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01077v3-figure-a737e669-997b-479a-81bc-d394556f031f.jpg)

The loss function combined time-domain loss ( L Time) and frequency-domain loss ( L Freq ) for robust learning:

$$\mathcal { L } _ { \text{overall} } = L _ { \text{Time} } + L _ { \text{Freq} },$$

where L Time minimizes the negative Pearson correlation between the predicted and ground-truth rPPG signals, and L Freq aligns their frequency-domain representations.

## V. EXPERIMENTAL EVALUATION

To fully verify PhysMamba's effectiveness and superiority in rPPG heart rate estimation, we have designed a detailed evaluation process, including both intra-dataset and crossdataset assessments. We compared PhysMamba with Unsupervised methods including GREEN [16], ICA [18], CHROM [17], LGI [19], PBV [20], and POS [21], OMIT [22], as well as state-of-the-art deep learning methods including DeepPhys [23], PhysNet [24], TS-CAN [26], PhysFormer [29],EfficientPhys [30], and RhythmMamba [10]. We provide additional experimental details in the supplementary materials.

## A. Intra-Dataset Testing

In intra-dataset evaluations, PhysMamba was trained and tested on the same dataset to assess its performance under consistent conditions. As shown in Table I, PhysMamba achieved near-optimal results on UBFC-rPPG and PURE datasets, with MAEs of 0.45 bpm and 0.24 bpm, respectively, outperforming existing methods. On the challenging MMPD dataset, PhysMamba demonstrated superior robustness, achieving an MAE of 2.84 bpm and the highest SNR of 5.20. These results indicate PhysMamba's ability to handle noise, motion artifacts, and varying lighting conditions effectively.The low computational complexity of PhysMamba, enabled by the SSSD framework, makes it particularly suitable for deployment on mobile devices and real-time monitoring systems. Its robustness on MMPD highlights its capability to adapt to complex, noisy environments.

## B. Cross-Dataset Testing

We selected the PURE dataset with head movements and the MMPD dataset with significant noise. Cross-dataset evaluations assessed PhysMamba's generalization by training and testing on different datasets, a crucial factor for real-world applications where training and deployment data often differ.

As shown in Tables II and III, PhysMamba consistently outperformed existing methods in cross-dataset scenarios. When trained on MMPD and tested on PURE, it achieved an MAE of 5.32 bpm, surpassing RhythmMamba. Conversely, when trained on PURE and tested on MMPD, it achieved an MAE of 9.87 bpm, significantly better than other methods.

The superior generalization of PhysMamba can be attributed to: SSSD : By combining state space models with attention mechanisms between the two pathways, PhysMamba effectively captures both long-term dependencies and shortterm variations. This dual capability enhances the model's robustness against dataset shifts caused by lighting, skin tone, and motion variability. Multi-Scale Queries : MQ enables efficient feature sharing across different time scales, improving the model's ability to adapt to diverse data distributions. This is particularly evident in the cross-dataset scenarios, where PhysMamba maintains high accuracy despite significant domain shifts. Practical Implications: The ability to generalize across datasets demonstrates PhysMamba's potential for deployment in real-world environments with minimal retraining. This is especially valuable for applications like remote health monitoring, where training data may not cover all possible variations in deployment settings.

## C. Ablation Study

To evaluate the contribution of each key component, we conducted ablation studies on the MMPD dataset. The results, summarized in Table IV, highlight the importance of the MQ, FDF, SSD, and CA modules in achieving state-of-theart performance.

Multi-Scale Queries (MQ): Improved the model's ability to capture multi-scale dependencies, reducing the MAE from 4.40 to 3.46 bpm. Frequency Domain Feed-Forward (FDF):

Fig. 5. Heat map of cross-dataset results: Training on MMPD and testing on PURE (left), and vice versa (right). PhysMamba shows superior generalization compared to other methods.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01077v3-figure-cf1a39a8-8450-4984-ae31-d1d6a37c56dd.jpg)

TABLE II

CROSS-DATASET RESULTS: TRAINED ON MMPD, TESTED ON PURE.

| Method             |   MAE |   RMSE |   MAPE |     r |
|--------------------|-------|--------|--------|-------|
| DeepPhys [23]      | 15.86 |  25.72 |  20.76 | -0.01 |
| PhysNet [24]       | 10.96 |  20.24 |  18.34 |  0.54 |
| TS-CAN [26]        |  7.24 |  16.8  |   9.76 |  0.68 |
| PhysFormer [29]    | 17.6  |  24.75 |  26.93 |  0    |
| EfficientPhys [30] |  5.68 |  16.59 |   6.26 |  0.7  |
| RhythmMamba [10]   |  6.07 |  14.43 |   9.66 |  0.78 |
| PhysMamba (Ours)   |  5.32 |  12.98 |   9.56 |  0.84 |

Enhanced the detection of periodic patterns, increasing SNR from 1.50 to 5.20. Dual-Pathway Design (DP): Reduced redundancy and improved feature learning, achieving an MAE of 3.13 bpm. SSSD Framework: Enabled robust temporal learning, ensuring the best overall performance with an MAE of 2.84 bpm and an SNR of 5.20. Practical Implications: The ablation study underscores the complementary roles of PhysMamba's components, particularly MQ and FDF, in balancing computational efficiency with robustness, making the model ideal for resource-constrained environments.

## D. Visualization and Further Analysis

To further analyze PhysMamba's performance, Figure 3 compares the predicted and ground-truth heart rate distributions across the three datasets. The results show a close alignment, even in challenging conditions like MMPD. In Figure 4, we present the visualization of the MMPD results, showing a strong correlation between the predicted and the ground truth heart rate.A second-order Butterworth filter (cutoff frequencies: 0.75 and 2.5 Hz) filtered the predicted PPG waveform. This result confirms PhysMamba's accuracy and reliability in heart rate estimation, even in challenging scenarios, demonstrating strong robustness. Figure 5 highlights the crossdataset results, showing PhysMamba's consistent performance across different training and testing conditions. This further validates its robustness and generalization capabilities. These visualizations reinforce the potential of PhysMamba for realworld applications.

## VI. CONCLUSION

This paper introduces PhysMamba, an efficient rPPG estimation method using State Space Duality in a dual-pathway

TABLE III

CROSS-DATASET RESULTS: TRAINED ON PURE, TESTED ON MMPD.

| Method             |   MAE |   RMSE |   MAPE |    r |
|--------------------|-------|--------|--------|------|
| DeepPhys [23]      | 16.92 |  24.61 |  18.54 | 0.05 |
| PhysNet [24]       | 13.94 |  21.61 |  15.15 | 0.2  |
| TS-CAN [26]        | 13.94 |  21.61 |  15.15 | 0.2  |
| PhysFormer [29]    | 14.57 |  20.71 |  16.73 | 0.15 |
| EfficientPhys [30] | 14.03 |  21.62 |  15.32 | 0.17 |
| RhythmMamba [10]   | 10.45 |  16.49 |  12.2  | 0.37 |
| PhysMamba (Ours)   |  9.87 |  15.47 |  11.76 | 0.4  |

## TABLE IV

ABLATION STUDY RESULTS ON THE MMPD DATASET. MQ: MULTI-SCALE

QUERIES, FDF: FREQUENCY DOMAIN FEED-FORWARD, DP:

DUAL-PATHWAY, SSD: STATE SPACE DUALITY, CA: CROSS-ATTENTION.

| MQ   | FDF   | DP   | SSD   | CA   |   MAE |   RMSE |   MAPE |    r |   SNR |
|------|-------|------|-------|------|-------|--------|--------|------|-------|
| ✓    | ✓     |      |       |      |  4.4  |   8.81 |   4.66 | 0.76 |  3.36 |
| ✓    | ✓     |      | ✓     |      |  3.46 |   7.21 |   3.69 | 0.84 |  3.94 |
| ✓    | ✓     | ✓    |       |      |  3.13 |   6.55 |   3.31 | 0.87 |  4.63 |
| ✓    | ✓     | ✓    | ✓     |      |  3.72 |   8.06 |   3.95 | 0.8  |  4.15 |
| ✓    |       | ✓    | ✓     | ✓    |  3.53 |   7.46 |   3.83 | 0.82 |  1.5  |
|      | ✓     | ✓    | ✓     | ✓    |  3.49 |   7.35 |   3.69 | 0.83 |  4.17 |
| ✓    | ✓     | ✓    | ✓     | ✓    |  2.84 |   6.41 |   3.04 | 0.88 |  5.2  |

time-frequency network. It combines long-term dependency modeling with a full-sequence attention mechanism. We also propose a simple yet innovative Synergistic State Space Duality algorithm with Multi-Scale Queries, facilitating effective information interaction within the dual-pathway network. Both internal and cross-dataset tests show that PhysMamba achieves state-of-the-art performance, with high accuracy and robustness, indicating strong potential in real-world physiological and psychological applications.

Future work will focus on extending PhysMamba for realtime monitoring of psychological states, emotions, and overall health in dynamic and diverse environments. With its high efficiency and adaptability, PhysMamba provides a scalable foundation for advancing non-contact health monitoring technologies, benefiting both individual well-being and public healthcare systems.

## REFERENCES

- [1] J.-H. Choi, K.-B. Kang, and K.-T. Kim, 'Fusion-vital: Video-rf fusion transformer for advanced remote physiological measurement,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 2, 2024, pp. 1344-1352.
- [2] X. Li, J. Chen, G. Zhao, and M. Pietikainen, 'Remote heart rate measurement from face videos under realistic situations,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2014, pp. 4264-4271.
- [3] S. Tulyakov, X. Alameda-Pineda, E. Ricci, L. Yin, J. F. Cohn, and N. Sebe, 'Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 23962404.
- [4] K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu et al. , 'A survey on vision transformer,' IEEE transactions on pattern analysis and machine intelligence , vol. 45, no. 1, pp. 87-110, 2022.
- [5] R. Xu, S. Yang, Y. Wang, B. Du, and H. Chen, 'A survey on vision mamba: Models, applications and challenges,' arXiv preprint arXiv:2404.18861 , 2024.
- [6] A. Gu and T. Dao, 'Mamba: Linear-time sequence modeling with selective state spaces,' arXiv preprint arXiv:2312.00752 , 2023.
- [7] S. Ma, Y. Kang, P. Bai, and Y.-B. Zhao, 'Fmamba: Mamba based on fast-attention for multivariate time-series forecasting,' arXiv preprint arXiv:2407.14814 , 2024.
- [8] J. Ma, F. Li, and B. Wang, 'U-mamba: Enhancing long-range dependency for biomedical image segmentation,' arXiv preprint arXiv:2401.04722 , 2024.
- [9] K. Li, X. Li, Y. Wang, Y. He, Y. Wang, L. Wang, and Y. Qiao, 'Videomamba: State space model for efficient video understanding,' arXiv preprint arXiv:2403.06977 , 2024.
- [10] B. Zou, Z. Guo, X. Hu, and H. Ma, 'Rhythmmamba: Fast remote physiological measurement with arbitrary length videos,' arXiv preprint arXiv:2404.06483 , 2024.
- [11] T. Dao and A. Gu, 'Transformers are ssms: Generalized models and efficient algorithms through structured state space duality,' arXiv preprint arXiv:2405.21060 , 2024.
- [12] X. Niu, Z. Yu, H. Han, X. Li, S. Shan, and G. Zhao, 'Video-based remote physiological measurement via cross-verified feature disentangling,' in Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16 . Springer, 2020, pp. 295-310.
- [13] K. He, X. Zhang, S. Ren, and J. Sun, 'Deep residual learning for image recognition,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770-778.
- [14] K. Cho, B. Van Merri¨ enboer, D. Bahdanau, and Y. Bengio, 'On the properties of neural machine translation: Encoder-decoder approaches,' arXiv preprint arXiv:1409.1259 , 2014.
- [15] Z. Yu, X. Li, X. Niu, J. Shi, and G. Zhao, 'Autohr: A strong end-toend baseline for remote heart rate measurement with neural searching,' IEEE Signal Processing Letters , vol. 27, pp. 1245-1249, 2020.
- [16] W. Verkruysse, L. O. Svaasand, and J. S. Nelson, 'Remote plethysmographic imaging using ambient light.' Optics express , vol. 16, no. 26, pp. 21 434-21 445, 2008.
- [17] G. De Haan and V. Jeanne, 'Robust pulse rate from chrominance-based rppg,' IEEE transactions on biomedical engineering , vol. 60, no. 10, pp. 2878-2886, 2013.
- [18] M.-Z. Poh, D. J. McDuff, and R. W. Picard, 'Non-contact, automated cardiac pulse measurements using video imaging and blind source separation.' Optics express , vol. 18, no. 10, pp. 10 762-10 774, 2010.
- [19] C. S. Pilz, S. Zaunseder, J. Krajewski, and V. Blazek, 'Local group invariance for heart rate estimation from face videos in the wild,' in Proceedings of the IEEE conference on computer vision and pattern recognition workshops , 2018, pp. 1254-1262.
- [20] G. De Haan and A. Van Leest, 'Improved motion robustness of remote-ppg by using the blood volume pulse signature,' Physiological measurement , vol. 35, no. 9, p. 1913, 2014.
- [21] W. Wang, A. C. Den Brinker, S. Stuijk, and G. De Haan, 'Algorithmic principles of remote ppg,' IEEE Transactions on Biomedical Engineering , vol. 64, no. 7, pp. 1479-1491, 2016.
- [22] C. A. Casado and M. B. L´ opez, 'Face2ppg: An unsupervised pipeline for blood volume pulse extraction from faces,' IEEE Journal of Biomedical and Health Informatics , 2023.
- [23] W. Chen and D. McDuff, 'Deepphys: Video-based physiological measurement using convolutional attention networks,' in Proceedings of the european conference on computer vision (ECCV) , 2018, pp. 349-365.
- [24] Z. Yu, X. Li, and G. Zhao, 'Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks,' arXiv preprint arXiv:1905.02419 , 2019.
- [25] X. Niu, H. Han, S. Shan, and X. Chen, 'Synrhythm: Learning a deep heart rate estimator from general to specific,' in 2018 24th international conference on pattern recognition (ICPR) . IEEE, 2018, pp. 3580-3585.
- [26] X. Liu, J. Fromm, S. Patel, and D. McDuff, 'Multi-task temporal shift attention networks for on-device contactless vitals measurement,' Advances in Neural Information Processing Systems , vol. 33, pp. 19 40019 411, 2020.
- [27] H. Lu, H. Han, and S. K. Zhou, 'Dual-gan: Joint bvp and noise modeling for remote physiological measurement,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 12 404-12 413.
- [28] J. Comas, A. Ruiz, and F. Sukno, 'Efficient remote photoplethysmography with temporal derivative modules and time-shift invariant loss,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 2182-2191.
- [29] Z. Yu, Y. Shen, J. Shi, H. Zhao, P. H. Torr, and G. Zhao, 'Physformer: Facial video-based physiological measurement with temporal difference transformer,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 4186-4196.
- [30] X. Liu, B. Hill, Z. Jiang, S. Patel, and D. McDuff, 'Efficientphys: Enabling simple, fast and accurate camera-based cardiac measurement,' in Proceedings of the IEEE/CVF winter conference on applications of computer vision , 2023, pp. 5008-5017.
- [31] R. Stricker, S. M¨ller, and H.-M. Gross, 'Non-contact video-based pulse u rate measurement on a mobile service robot,' in The 23rd IEEE International Symposium on Robot and Human Interactive Communication . IEEE, 2014, pp. 1056-1062.
- [32] S. Bobbia, R. Macwan, Y. Benezeth, A. Mansouri, and J. Dubois, 'Unsupervised skin tissue segmentation for remote photoplethysmography,' Pattern Recognition Letters , vol. 124, pp. 82-90, 2019.
- [33] J. Tang, K. Chen, Y. Wang, Y. Shi, S. Patel, D. McDuff, and X. Liu, 'Mmpd: multi-domain mobile video physiology dataset,' in 2023 45th Annual International Conference of the IEEE Engineering in Medicine &amp; Biology Society (EMBC) . IEEE, 2023, pp. 1-5.
- [34] X. Liu, G. Narayanswamy, A. Paruchuri, X. Zhang, J. Tang, Y. Zhang, R. Sengupta, S. Patel, Y. Wang, and D. McDuff, 'rppg-toolbox: Deep remote ppg toolbox,' Advances in Neural Information Processing Systems , vol. 36, 2024.
Zhixin Yan Yan Zhong Hongbin Xu Wenjun Zhang Shangru Yi Lin Shu Wenxiong Kang
PhysMamba: State Space Duality Model for Remote Physiological Measurement
Remote Photoplethysmography (rPPG) enables non-contact physiological signal
extraction from facial videos, offering applications in psychological state
analysis, medical assistance, and anti-face spoofing. However, challenges such
as motion artifacts, lighting variations, and noise limit its real-world
applicability. To address these issues, we propose PhysMamba, a novel
dual-pathway time-frequency interaction model based on Synergistic State Space
Duality (SSSD), which for the first time integrates state space models with
attention mechanisms in a dual-branch framework. Combined with a Multi-Scale
Query (MQ) mechanism, PhysMamba achieves efficient information exchange and
enhanced feature representation, ensuring robustness under noisy and dynamic
conditions. Experiments on PURE, UBFC-rPPG, and MMPD datasets demonstrate that
PhysMamba outperforms state-of-the-art methods, offering superior accuracy and
generalization. This work lays a strong foundation for practical applications
in non-contact health monitoring, including real-time remote patient care.