title: Boosting Healthcare LLMs Through Retrieved Context
summary: Large Language Models (LLMs) have demonstrated remarkable capabilities in
  natural language processing, and yet, their factual inaccuracies and
  hallucinations limits their application, particularly in critical domains like
  healthcare. Context retrieval methods, by introducing relevant information as
  input, have emerged as a crucial approach for enhancing LLM factuality and
  reliability. This study explores the boundaries of context retrieval methods
  within the healthcare domain, optimizing their components and benchmarking
  their performance against open and closed alternatives. Our findings reveal how
  open LLMs, when augmented with an optimized retrieval system, can achieve
  performance comparable to the biggest private solutions on established
  healthcare benchmarks (multiple-choice question answering). Recognizing the
  lack of realism of including the possible answers within the question (a setup
  only found in medical exams), and after assessing a strong LLM performance
  degradation in the absence of those options, we extend the context retrieval
  system in that direction. In particular, we propose OpenMedPrompt a pipeline
  that improves the generation of more reliable open-ended answers, moving this
  technology closer to practical application.
paper_authors._id: 66f50e15aa828de5c32d47aa
paper_authors.name: Jordi Bayarri-Planas
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-26T09:17:31.011Z
paper_authors._id: 6620f941eba5274b5c12f83d
paper_authors.avatarUrl: /avatars/55109cf647c38499b471a05e3dd04ba3.svg
paper_authors.fullname: Jordi Bayarri
paper_authors.type: user
paper_authors.user: JordiBayarri
paper_authors._id: 66f50e15aa828de5c32d47ab
paper_authors.name: Ashwin Kumar Gururajan
paper_authors._id: 66f50e15aa828de5c32d47ac
paper_authors.name: Dario Garcia-Gasulla
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-09-26T07:52:49.778Z
paper_authors._id: 62f7a16192950415b637e201
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62f7a16192950415b637e201/4IIqYap43vujvSuql68Vj.jpeg
paper_authors.fullname: Dario
paper_authors.type: user
paper_authors.user: dariog
paper_title: Boosting Healthcare LLMs Through Retrieved Context
paper_summary: Large Language Models (LLMs) have demonstrated remarkable capabilities in
  natural language processing, and yet, their factual inaccuracies and
  hallucinations limits their application, particularly in critical domains like
  healthcare. Context retrieval methods, by introducing relevant information as
  input, have emerged as a crucial approach for enhancing LLM factuality and
  reliability. This study explores the boundaries of context retrieval methods
  within the healthcare domain, optimizing their components and benchmarking
  their performance against open and closed alternatives. Our findings reveal how
  open LLMs, when augmented with an optimized retrieval system, can achieve
  performance comparable to the biggest private solutions on established
  healthcare benchmarks (multiple-choice question answering). Recognizing the
  lack of realism of including the possible answers within the question (a setup
  only found in medical exams), and after assessing a strong LLM performance
  degradation in the absence of those options, we extend the context retrieval
  system in that direction. In particular, we propose OpenMedPrompt a pipeline
  that improves the generation of more reliable open-ended answers, moving this
  technology closer to practical application.
paper_ai_keywords: Language Models (LLMs) Natural language processing Factual inaccuracies Hallucinations Context retrieval methods Healthcare Optimization Benchmarking Multiple-choice question answering OpenMedPrompt Open-ended answers