title: Towards Conversational Diagnostic AI
summary: At the heart of medicine lies the physician-patient dialogue, where skillful
  history-taking paves the way for accurate diagnosis, effective management, and
  enduring trust. Artificial Intelligence (AI) systems capable of diagnostic
  dialogue could increase accessibility, consistency, and quality of care.
  However, approximating clinicians' expertise is an outstanding grand challenge.
  Here, we introduce AMIE (Articulate Medical Intelligence Explorer), a Large
  Language Model (LLM) based AI system optimized for diagnostic dialogue.
    AMIE uses a novel self-play based simulated environment with automated
  feedback mechanisms for scaling learning across diverse disease conditions,
  specialties, and contexts. We designed a framework for evaluating
  clinically-meaningful axes of performance including history-taking, diagnostic
  accuracy, management reasoning, communication skills, and empathy. We compared
  AMIE's performance to that of primary care physicians (PCPs) in a randomized,
  double-blind crossover study of text-based consultations with validated patient
  actors in the style of an Objective Structured Clinical Examination (OSCE). The
  study included 149 case scenarios from clinical providers in Canada, the UK,
  and India, 20 PCPs for comparison with AMIE, and evaluations by specialist
  physicians and patient actors. AMIE demonstrated greater diagnostic accuracy
  and superior performance on 28 of 32 axes according to specialist physicians
  and 24 of 26 axes according to patient actors. Our research has several
  limitations and should be interpreted with appropriate caution. Clinicians were
  limited to unfamiliar synchronous text-chat which permits large-scale
  LLM-patient interactions but is not representative of usual clinical practice.
  While further research is required before AMIE could be translated to
  real-world settings, the results represent a milestone towards conversational
  diagnostic AI.
paper_authors._id: 65a0a488ff65c2d46d9efe22
paper_authors.name: Tao Tu
paper_authors._id: 65a0a488ff65c2d46d9efe23
paper_authors.name: Anil Palepu
paper_authors._id: 65a0a488ff65c2d46d9efe24
paper_authors.name: Mike Schaekermann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-12T10:09:36.391Z
paper_authors._id: 64ebb75f171e6c9862d261c9
paper_authors.avatarUrl: /avatars/b7750f5d6281f906cfa5c717a44e2e13.svg
paper_authors.fullname: Mike Schaekermann
paper_authors.type: user
paper_authors.user: mikeschaekermann
paper_authors._id: 65a0a488ff65c2d46d9efe25
paper_authors.name: Khaled Saab
paper_authors._id: 65a0a488ff65c2d46d9efe26
paper_authors.name: Jan Freyberg
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-12T10:09:47.258Z
paper_authors._id: 629656557d586f66c39206e7
paper_authors.avatarUrl: /avatars/91245f3b6a2eccebef872ed00bdfea73.svg
paper_authors.fullname: Jan Freyberg
paper_authors.type: user
paper_authors.user: jfreyberg
paper_authors._id: 65a0a488ff65c2d46d9efe27
paper_authors.name: Ryutaro Tanno
paper_authors._id: 65a0a488ff65c2d46d9efe28
paper_authors.name: Amy Wang
paper_authors._id: 65a0a488ff65c2d46d9efe29
paper_authors.name: Brenna Li
paper_authors._id: 65a0a488ff65c2d46d9efe2a
paper_authors.name: Mohamed Amin
paper_authors._id: 65a0a488ff65c2d46d9efe2b
paper_authors.name: Nenad Tomasev
paper_authors._id: 65a0a488ff65c2d46d9efe2c
paper_authors.name: Shekoofeh Azizi
paper_authors._id: 65a0a488ff65c2d46d9efe2d
paper_authors.name: Karan Singhal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-12T10:10:19.355Z
paper_authors._id: 654bb16a48b474120258371c
paper_authors.avatarUrl: /avatars/7ad4a33503533ddbb54c71b634fda89b.svg
paper_authors.fullname: Karan Singhal
paper_authors.type: user
paper_authors.user: karan1149
paper_authors._id: 65a0a488ff65c2d46d9efe2e
paper_authors.name: Yong Cheng
paper_authors._id: 65a0a488ff65c2d46d9efe2f
paper_authors.name: Le Hou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-12T10:10:44.869Z
paper_authors._id: 626b2402c804c432c1b5004c
paper_authors.avatarUrl: /avatars/a119644422335e260af0ca4ef6399870.svg
paper_authors.fullname: Le Hou
paper_authors.type: user
paper_authors.user: solooobear
paper_authors._id: 65a0a488ff65c2d46d9efe30
paper_authors.name: Albert Webson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-12T10:10:52.013Z
paper_authors._id: 617ada70ff3db6021d069798
paper_authors.avatarUrl: /avatars/f14c59fd56a49dd9a925285a4fb66fff.svg
paper_authors.fullname: Albert Webson
paper_authors.type: user
paper_authors.user: awebson
paper_authors._id: 65a0a488ff65c2d46d9efe31
paper_authors.name: Kavita Kulkarni
paper_authors._id: 65a0a488ff65c2d46d9efe32
paper_authors.name: S Sara Mahdavi
paper_authors._id: 65a0a488ff65c2d46d9efe33
paper_authors.name: Christopher Semturs
paper_authors._id: 65a0a488ff65c2d46d9efe34
paper_authors.name: Juraj Gottweis
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-12T10:11:10.554Z
paper_authors._id: 6381e100cae34727b9ca33ce
paper_authors.avatarUrl: /avatars/f4e047e82e41ec507c6f51aef7560d87.svg
paper_authors.fullname: Juraj Gottweis
paper_authors.type: user
paper_authors.user: juro
paper_authors._id: 65a0a488ff65c2d46d9efe35
paper_authors.name: Joelle Barral
paper_authors._id: 65a0a488ff65c2d46d9efe36
paper_authors.name: Katherine Chou
paper_authors._id: 65a0a488ff65c2d46d9efe37
paper_authors.name: Greg S Corrado
paper_authors._id: 65a0a488ff65c2d46d9efe38
paper_authors.name: Yossi Matias
paper_authors._id: 65a0a488ff65c2d46d9efe39
paper_authors.name: Alan Karthikesalingam
paper_authors._id: 65a0a488ff65c2d46d9efe3a
paper_authors.name: Vivek Natarajan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-12T10:11:37.514Z
paper_authors._id: 617a6af2941993035fbaf29c
paper_authors.avatarUrl: /avatars/a389671494782f06ff8502c934090cc0.svg
paper_authors.fullname: Vivek Natarajan
paper_authors.type: user
paper_authors.user: vivnat
paper_title: Towards Conversational Diagnostic AI
paper_summary: At the heart of medicine lies the physician-patient dialogue, where skillful
  history-taking paves the way for accurate diagnosis, effective management, and
  enduring trust. Artificial Intelligence (AI) systems capable of diagnostic
  dialogue could increase accessibility, consistency, and quality of care.
  However, approximating clinicians' expertise is an outstanding grand challenge.
  Here, we introduce AMIE (Articulate Medical Intelligence Explorer), a Large
  Language Model (LLM) based AI system optimized for diagnostic dialogue.
    AMIE uses a novel self-play based simulated environment with automated
  feedback mechanisms for scaling learning across diverse disease conditions,
  specialties, and contexts. We designed a framework for evaluating
  clinically-meaningful axes of performance including history-taking, diagnostic
  accuracy, management reasoning, communication skills, and empathy. We compared
  AMIE's performance to that of primary care physicians (PCPs) in a randomized,
  double-blind crossover study of text-based consultations with validated patient
  actors in the style of an Objective Structured Clinical Examination (OSCE). The
  study included 149 case scenarios from clinical providers in Canada, the UK,
  and India, 20 PCPs for comparison with AMIE, and evaluations by specialist
  physicians and patient actors. AMIE demonstrated greater diagnostic accuracy
  and superior performance on 28 of 32 axes according to specialist physicians
  and 24 of 26 axes according to patient actors. Our research has several
  limitations and should be interpreted with appropriate caution. Clinicians were
  limited to unfamiliar synchronous text-chat which permits large-scale
  LLM-patient interactions but is not representative of usual clinical practice.
  While further research is required before AMIE could be translated to
  real-world settings, the results represent a milestone towards conversational
  diagnostic AI.
paper_ai_keywords: Large Language Model (LLM) self-play based simulated environment automated feedback mechanisms diagnostic dialogue history-taking diagnostic accuracy management reasoning communication skills empathy randomized, double-blind crossover study Objective Structured Clinical Examination (OSCE) large-scale LLM-patient interactions