title: Teach Multimodal LLMs to Comprehend Electrocardiographic Images
summary: The electrocardiogram (ECG) is an essential non-invasive diagnostic tool for
  assessing cardiac conditions. Existing automatic interpretation methods suffer
  from limited generalizability, focusing on a narrow range of cardiac
  conditions, and typically depend on raw physiological signals, which may not be
  readily available in resource-limited settings where only printed or digital
  ECG images are accessible. Recent advancements in multimodal large language
  models (MLLMs) present promising opportunities for addressing these challenges.
  However, the application of MLLMs to ECG image interpretation remains
  challenging due to the lack of instruction tuning datasets and well-established
  ECG image benchmarks for quantitative evaluation. To address these challenges,
  we introduce ECGInstruct, a comprehensive ECG image instruction tuning dataset
  of over one million samples, covering a wide range of ECG-related tasks from
  diverse data sources. Using ECGInstruct, we develop PULSE, an MLLM tailored for
  ECG image comprehension. In addition, we curate ECGBench, a new evaluation
  benchmark covering four key ECG image interpretation tasks across nine
  different datasets. Our experiments show that PULSE sets a new
  state-of-the-art, outperforming general MLLMs with an average accuracy
  improvement of 15% to 30%. This work highlights the potential of PULSE to
  enhance ECG interpretation in clinical practice.
paper_authors._id: 671ee90c77035878c52e67b2
paper_authors.name: Ruoqi Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:18:19.340Z
paper_authors._id: 656fd85f039133e875431ed3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/656fd85f039133e875431ed3/_Fc8m0hk0MXOfi3470Kru.jpeg
paper_authors.fullname: Ruoqi Liu
paper_authors.type: user
paper_authors.user: RuoqiLiu
paper_authors._id: 671ee90c77035878c52e67b3
paper_authors.name: Yuelin Bai
paper_authors._id: 671ee90c77035878c52e67b4
paper_authors.name: Xiang Yue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:18:37.003Z
paper_authors._id: 6230d750d93e84e233882dbc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6230d750d93e84e233882dbc/4MGEekLW3oWzqeFWDWvIK.jpeg
paper_authors.fullname: Xiang Yue
paper_authors.type: user
paper_authors.user: yuexiang96
paper_authors._id: 671ee90c77035878c52e67b5
paper_authors.name: Ping Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-28T15:19:05.468Z
paper_authors._id: 671f96c070f231ff42ceadc6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/671f96c070f231ff42ceadc6/lIRWnc9-zItr6QoGs0N3u.jpeg
paper_authors.fullname: Ping Zhang
paper_authors.type: user
paper_authors.user: aidhlab
paper_title: Teach Multimodal LLMs to Comprehend Electrocardiographic Images
paper_summary: The electrocardiogram (ECG) is an essential non-invasive diagnostic tool for
  assessing cardiac conditions. Existing automatic interpretation methods suffer
  from limited generalizability, focusing on a narrow range of cardiac
  conditions, and typically depend on raw physiological signals, which may not be
  readily available in resource-limited settings where only printed or digital
  ECG images are accessible. Recent advancements in multimodal large language
  models (MLLMs) present promising opportunities for addressing these challenges.
  However, the application of MLLMs to ECG image interpretation remains
  challenging due to the lack of instruction tuning datasets and well-established
  ECG image benchmarks for quantitative evaluation. To address these challenges,
  we introduce ECGInstruct, a comprehensive ECG image instruction tuning dataset
  of over one million samples, covering a wide range of ECG-related tasks from
  diverse data sources. Using ECGInstruct, we develop PULSE, an MLLM tailored for
  ECG image comprehension. In addition, we curate ECGBench, a new evaluation
  benchmark covering four key ECG image interpretation tasks across nine
  different datasets. Our experiments show that PULSE sets a new
  state-of-the-art, outperforming general MLLMs with an average accuracy
  improvement of 15% to 30%. This work highlights the potential of PULSE to
  enhance ECG interpretation in clinical practice.
paper_ai_keywords: multimodal large language models (MLLMs) ECGInstruct ECG image instruction tuning dataset PULSE ECG image comprehension ECGBench evaluation benchmark ECG image interpretation tasks state-of-the-art