Multimodal Variational Autoencoder for Low-cost Cardiac Hemodynamics Instability Detection
<|reference_start|>Multimodal Variational Autoencoder for Low-cost Cardiac Hemodynamics Instability Detection: Recent advancements in non-invasive detection of cardiac hemodynamic instability (CHDI) primarily focus on applying machine learning techniques to a single data modality, e.g. cardiac magnetic resonance imaging (MRI). Despite their potential, these approaches often fall short especially when the size of labeled patient data is limited, a common challenge in the medical domain. Furthermore, only a few studies have explored multimodal methods to study CHDI, which mostly rely on costly modalities such as cardiac MRI and echocardiogram. In response to these limitations, we propose a novel multimodal variational autoencoder ($\text{CardioVAE}_\text{X,G}$) to integrate low-cost chest X-ray (CXR) and electrocardiogram (ECG) modalities with pre-training on a large unlabeled dataset. Specifically, $\text{CardioVAE}_\text{X,G}$ introduces a novel tri-stream pre-training strategy to learn both shared and modality-specific features, thus enabling fine-tuning with both unimodal and multimodal datasets. We pre-train $\text{CardioVAE}_\text{X,G}$ on a large, unlabeled dataset of $50,982$ subjects from a subset of MIMIC database and then fine-tune the pre-trained model on a labeled dataset of $795$ subjects from the ASPIRE registry. Comprehensive evaluations against existing methods show that $\text{CardioVAE}_\text{X,G}$ offers promising performance (AUROC $=0.79$ and Accuracy $=0.77$), representing a significant step forward in non-invasive prediction of CHDI. Our model also excels in producing fine interpretations of predictions directly associated with clinical features, thereby supporting clinical decision-making.<|reference_end|>
@article{suvon2024multimodal,
    title={Multimodal Variational Autoencoder for Low-cost Cardiac Hemodynamics
  Instability Detection},
    author={Mohammod N. I. Suvon, Prasun C. Tripathi, Wenrui Fan, Shuo Zhou,
  Xianyuan Liu, Samer Alabed, Venet Osmani, Andrew J. Swift, Chen Chen, Haiping
  Lu},
    journal={arXiv preprint arXiv:2403.13658},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.13658},
    primaryClass={cs.LG cs.CV}
}