title: Clinical Text Summarization: Adapting Large Language Models Can
    Outperform Human Experts
summary: Sifting through vast textual data and summarizing key information imposes a
  substantial burden on how clinicians allocate their time. Although large
  language models (LLMs) have shown immense promise in natural language
  processing (NLP) tasks, their efficacy across diverse clinical summarization
  tasks has not yet been rigorously examined. In this work, we employ domain
  adaptation methods on eight LLMs, spanning six datasets and four distinct
  summarization tasks: radiology reports, patient questions, progress notes, and
  doctor-patient dialogue. Our thorough quantitative assessment reveals
  trade-offs between models and adaptation methods in addition to instances where
  recent advances in LLMs may not lead to improved results. Further, in a
  clinical reader study with six physicians, we depict that summaries from the
  best adapted LLM are preferable to human summaries in terms of completeness and
  correctness. Our ensuing qualitative analysis delineates mutual challenges
  faced by both LLMs and human experts. Lastly, we correlate traditional
  quantitative NLP metrics with reader study scores to enhance our understanding
  of how these metrics align with physician preferences. Our research marks the
  first evidence of LLMs outperforming human experts in clinical text
  summarization across multiple tasks. This implies that integrating LLMs into
  clinical workflows could alleviate documentation burden, empowering clinicians
  to focus more on personalized patient care and other irreplaceable human
  aspects of medicine.
paper_authors._id: 6503bd65c6ce94dc0e1dd13c
paper_authors.name: Dave Van Veen
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-09-15T05:59:55.991Z
paper_authors._id: 64e36dd9618cd90997e048f2
paper_authors.avatarUrl: /avatars/2a239bed193982596114d52afea1bb97.svg
paper_authors.fullname: Dave Van Veen
paper_authors.type: user
paper_authors.user: davevanveen
paper_authors._id: 6503bd65c6ce94dc0e1dd13d
paper_authors.name: Cara Van Uden
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T07:57:13.219Z
paper_authors._id: 64c7c84fe761f47061035533
paper_authors.avatarUrl: /avatars/810b73fd2ca0f6e7028a84496e542e91.svg
paper_authors.fullname: Cara Van Uden
paper_authors.type: user
paper_authors.user: cvanuden
paper_authors._id: 6503bd65c6ce94dc0e1dd13e
paper_authors.name: Louis Blankemeier
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T07:53:03.851Z
paper_authors._id: 6377044ea3b787faca443cc0
paper_authors.avatarUrl: /avatars/5cd6cc70ad597dd83776018a452947d0.svg
paper_authors.fullname: Louis Blankemeier
paper_authors.type: user
paper_authors.user: louisblankemeier
paper_authors._id: 6503bd65c6ce94dc0e1dd13f
paper_authors.name: Jean-Benoit Delbrouck
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-18T09:10:58.982Z
paper_authors._id: 62716952bcef985363db8485
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62716952bcef985363db8485/zJPPo5xlwZRJdEuwYsYKp.jpeg
paper_authors.fullname: JB D.
paper_authors.type: user
paper_authors.user: IAMJB
paper_authors._id: 6503bd65c6ce94dc0e1dd140
paper_authors.name: Asad Aali
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T07:57:59.027Z
paper_authors._id: 64bac7c5e38420aaba8ea197
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/jU-MCO7qQ5DE_o46Br57v.png
paper_authors.fullname: Asad Aali
paper_authors.type: user
paper_authors.user: asadaali
paper_authors._id: 6503bd65c6ce94dc0e1dd141
paper_authors.name: Christian Bluethgen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T07:54:08.286Z
paper_authors._id: 60c540b2fcb103f4f6a53c5a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1653944246103-60c540b2fcb103f4f6a53c5a.jpeg
paper_authors.fullname: Christian Bluethgen
paper_authors.type: user
paper_authors.user: pogchamp
paper_authors._id: 6503bd65c6ce94dc0e1dd142
paper_authors.name: Anuj Pareek
paper_authors._id: 6503bd65c6ce94dc0e1dd143
paper_authors.name: Malgorzata Polacin
paper_authors._id: 6503bd65c6ce94dc0e1dd144
paper_authors.name: William Collins
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T08:00:55.986Z
paper_authors._id: 642fd225450c0de9a1d1b641
paper_authors.avatarUrl: /avatars/7b457f909e94cda3875884381bcd8e39.svg
paper_authors.fullname: William Collins
paper_authors.type: user
paper_authors.user: wwcollins
paper_authors._id: 6503bd65c6ce94dc0e1dd145
paper_authors.name: Neera Ahuja
paper_authors._id: 6503bd65c6ce94dc0e1dd146
paper_authors.name: Curtis P. Langlotz
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T07:55:50.293Z
paper_authors._id: 62867edd1d99648808888d7b
paper_authors.avatarUrl: /avatars/22d764782c5169634deec6f43ece7500.svg
paper_authors.fullname: Curt Langlotz
paper_authors.type: user
paper_authors.user: cplanglotz
paper_authors._id: 6503bd65c6ce94dc0e1dd147
paper_authors.name: Jason Hom
paper_authors._id: 6503bd65c6ce94dc0e1dd148
paper_authors.name: Sergios Gatidis
paper_authors._id: 6503bd65c6ce94dc0e1dd149
paper_authors.name: John Pauly
paper_authors._id: 6503bd65c6ce94dc0e1dd14a
paper_authors.name: Akshay S. Chaudhari
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T07:57:03.094Z
paper_authors._id: 6236533b76c8a780323af640
paper_authors.avatarUrl: /avatars/18078ad26aa44312a4927160216e5943.svg
paper_authors.fullname: Akshay Chaudhari
paper_authors.type: user
paper_authors.user: akshaysc
paper_title: Clinical Text Summarization: Adapting Large Language Models Can
    Outperform Human Experts
paper_summary: Sifting through vast textual data and summarizing key information imposes a
  substantial burden on how clinicians allocate their time. Although large
  language models (LLMs) have shown immense promise in natural language
  processing (NLP) tasks, their efficacy across diverse clinical summarization
  tasks has not yet been rigorously examined. In this work, we employ domain
  adaptation methods on eight LLMs, spanning six datasets and four distinct
  summarization tasks: radiology reports, patient questions, progress notes, and
  doctor-patient dialogue. Our thorough quantitative assessment reveals
  trade-offs between models and adaptation methods in addition to instances where
  recent advances in LLMs may not lead to improved results. Further, in a
  clinical reader study with six physicians, we depict that summaries from the
  best adapted LLM are preferable to human summaries in terms of completeness and
  correctness. Our ensuing qualitative analysis delineates mutual challenges
  faced by both LLMs and human experts. Lastly, we correlate traditional
  quantitative NLP metrics with reader study scores to enhance our understanding
  of how these metrics align with physician preferences. Our research marks the
  first evidence of LLMs outperforming human experts in clinical text
  summarization across multiple tasks. This implies that integrating LLMs into
  clinical workflows could alleviate documentation burden, empowering clinicians
  to focus more on personalized patient care and other irreplaceable human
  aspects of medicine.
paper_ai_keywords: large language models (LLMs) natural language processing (NLP) domain adaptation summarization tasks radiology reports patient questions progress notes doctor-patient dialogue clinical reader study completeness correctness traditional quantitative NLP metrics physician preferences clinical text summarization