title: EchoPrime: A Multi-Video View-Informed Vision-Language Model for
    Comprehensive Echocardiography Interpretation
summary: Echocardiography is the most widely used cardiac imaging modality, capturing
  ultrasound video data to assess cardiac structure and function. Artificial
  intelligence (AI) in echocardiography has the potential to streamline manual
  tasks and improve reproducibility and precision. However, most echocardiography
  AI models are single-view, single-task systems that do not synthesize
  complementary information from multiple views captured during a full exam, and
  thus lead to limited performance and scope of applications. To address this
  problem, we introduce EchoPrime, a multi-view, view-informed, video-based
  vision-language foundation model trained on over 12 million video-report pairs.
  EchoPrime uses contrastive learning to train a unified embedding model for all
  standard views in a comprehensive echocardiogram study with representation of
  both rare and common diseases and diagnoses. EchoPrime then utilizes
  view-classification and a view-informed anatomic attention model to weight
  video-specific interpretations that accurately maps the relationship between
  echocardiographic views and anatomical structures. With retrieval-augmented
  interpretation, EchoPrime integrates information from all echocardiogram videos
  in a comprehensive study and performs holistic comprehensive clinical
  echocardiography interpretation. In datasets from two independent healthcare
  systems, EchoPrime achieves state-of-the art performance on 23 diverse
  benchmarks of cardiac form and function, surpassing the performance of both
  task-specific approaches and prior foundation models. Following rigorous
  clinical evaluation, EchoPrime can assist physicians in the automated
  preliminary assessment of comprehensive echocardiography.
paper_authors._id: 670ee7064d88c510f47ca128
paper_authors.name: Milos Vukadinovic
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-16T09:44:12.113Z
paper_authors._id: 64fb4eca0e486522f81fba18
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/OKC0X55IYBsDfVJhDyc5v.png
paper_authors.fullname: Milos Vukadinovic
paper_authors.type: user
paper_authors.user: milosvuk
paper_authors._id: 670ee7064d88c510f47ca129
paper_authors.name: Xiu Tang
paper_authors._id: 670ee7064d88c510f47ca12a
paper_authors.name: Neal Yuan
paper_authors._id: 670ee7064d88c510f47ca12b
paper_authors.name: Paul Cheng
paper_authors._id: 670ee7064d88c510f47ca12c
paper_authors.name: Debiao Li
paper_authors._id: 670ee7064d88c510f47ca12d
paper_authors.name: Susan Cheng
paper_authors._id: 670ee7064d88c510f47ca12e
paper_authors.name: Bryan He
paper_authors._id: 670ee7064d88c510f47ca12f
paper_authors.name: David Ouyang
paper_title: EchoPrime: A Multi-Video View-Informed Vision-Language Model for
    Comprehensive Echocardiography Interpretation
paper_summary: Echocardiography is the most widely used cardiac imaging modality, capturing
  ultrasound video data to assess cardiac structure and function. Artificial
  intelligence (AI) in echocardiography has the potential to streamline manual
  tasks and improve reproducibility and precision. However, most echocardiography
  AI models are single-view, single-task systems that do not synthesize
  complementary information from multiple views captured during a full exam, and
  thus lead to limited performance and scope of applications. To address this
  problem, we introduce EchoPrime, a multi-view, view-informed, video-based
  vision-language foundation model trained on over 12 million video-report pairs.
  EchoPrime uses contrastive learning to train a unified embedding model for all
  standard views in a comprehensive echocardiogram study with representation of
  both rare and common diseases and diagnoses. EchoPrime then utilizes
  view-classification and a view-informed anatomic attention model to weight
  video-specific interpretations that accurately maps the relationship between
  echocardiographic views and anatomical structures. With retrieval-augmented
  interpretation, EchoPrime integrates information from all echocardiogram videos
  in a comprehensive study and performs holistic comprehensive clinical
  echocardiography interpretation. In datasets from two independent healthcare
  systems, EchoPrime achieves state-of-the art performance on 23 diverse
  benchmarks of cardiac form and function, surpassing the performance of both
  task-specific approaches and prior foundation models. Following rigorous
  clinical evaluation, EchoPrime can assist physicians in the automated
  preliminary assessment of comprehensive echocardiography.
paper_ai_keywords: echocardiography AI models multi-view view-informed video-based vision-language foundation model contrastive learning unified embedding model view-classification view-informed attention model retrieval-augmented interpretation holistic clinical echocardiography interpretation state-of-the-art