title: How do you know that? Teaching Generative Language Models to Reference
    Answers to Biomedical Questions
summary: Large language models (LLMs) have recently become the leading source of
  answers for users' questions online. Despite their ability to offer eloquent
  answers, their accuracy and reliability can pose a significant challenge. This
  is especially true for sensitive domains such as biomedicine, where there is a
  higher need for factually correct answers. This paper introduces a biomedical
  retrieval-augmented generation (RAG) system designed to enhance the reliability
  of generated responses. The system is based on a fine-tuned LLM for the
  referenced question-answering, where retrieved relevant abstracts from PubMed
  are passed to LLM's context as input through a prompt. Its output is an answer
  based on PubMed abstracts, where each statement is referenced accordingly,
  allowing the users to verify the answer. Our retrieval system achieves an
  absolute improvement of 23% compared to the PubMed search engine. Based on the
  manual evaluation on a small sample, our fine-tuned LLM component achieves
  comparable results to GPT-4 Turbo in referencing relevant abstracts. We make
  the dataset used to fine-tune the models and the fine-tuned models based on
  Mistral-7B-instruct-v0.1 and v0.2 publicly available.
paper_authors._id: 668d0678fecd7e04836275d2
paper_authors.name: Bojana Bašaragin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:50:35.775Z
paper_authors._id: 627b83b37a5a8a708ffd18d9
paper_authors.avatarUrl: /avatars/6e379274f82f343c94fde99e0a74fd81.svg
paper_authors.fullname: Bojana Bašaragin
paper_authors.type: user
paper_authors.user: BojanaBas
paper_authors._id: 668d0678fecd7e04836275d3
paper_authors.name: Adela Ljajić
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:50:42.133Z
paper_authors._id: 627da21cba19b19eff696abe
paper_authors.avatarUrl: /avatars/870ae4ea50c5c9ef31ec9e772e068995.svg
paper_authors.fullname: Adela Ljajić
paper_authors.type: user
paper_authors.user: Adelija
paper_authors._id: 668d0678fecd7e04836275d4
paper_authors.name: Darija Medvecki
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:50:47.573Z
paper_authors._id: 64751547b68461d5cf7d5aec
paper_authors.avatarUrl: /avatars/909ab45955970f5ff47127262210022e.svg
paper_authors.fullname: DarijaM
paper_authors.type: user
paper_authors.user: DarijaM
paper_authors._id: 668d0678fecd7e04836275d5
paper_authors.name: Lorenzo Cassano
paper_authors._id: 668d0678fecd7e04836275d6
paper_authors.name: Miloš Košprdić
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:51:12.705Z
paper_authors._id: 6408811a92033c15073b4d08
paper_authors.avatarUrl: /avatars/847c126d0b4fa0834776d73805e19d37.svg
paper_authors.fullname: Miloš Košprdić
paper_authors.type: user
paper_authors.user: MilosKosRad
paper_authors._id: 668d0678fecd7e04836275d7
paper_authors.name: Nikola Milošević
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-10T08:20:00.186Z
paper_authors._id: 6204dcc2522e40b4a18d86eb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644485808126-noauth.jpeg
paper_authors.fullname: Nikola
paper_authors.type: user
paper_authors.user: nikolamilosevic
paper_title: How do you know that? Teaching Generative Language Models to Reference
    Answers to Biomedical Questions
paper_summary: Large language models (LLMs) have recently become the leading source of
  answers for users' questions online. Despite their ability to offer eloquent
  answers, their accuracy and reliability can pose a significant challenge. This
  is especially true for sensitive domains such as biomedicine, where there is a
  higher need for factually correct answers. This paper introduces a biomedical
  retrieval-augmented generation (RAG) system designed to enhance the reliability
  of generated responses. The system is based on a fine-tuned LLM for the
  referenced question-answering, where retrieved relevant abstracts from PubMed
  are passed to LLM's context as input through a prompt. Its output is an answer
  based on PubMed abstracts, where each statement is referenced accordingly,
  allowing the users to verify the answer. Our retrieval system achieves an
  absolute improvement of 23% compared to the PubMed search engine. Based on the
  manual evaluation on a small sample, our fine-tuned LLM component achieves
  comparable results to GPT-4 Turbo in referencing relevant abstracts. We make
  the dataset used to fine-tune the models and the fine-tuned models based on
  Mistral-7B-instruct-v0.1 and v0.2 publicly available.
paper_ai_summary: A biomedical retrieval-augmented generation system enhances the reliability of LLM-generated responses by integrating referenced PubMed abstracts, achieving improved accuracy and reference quality compared to existing methods.
paper_ai_keywords: LLMs retrieval-augmented generation biomedical domain PubMed fine-tuned LLM GPT-4 Turbo Mistral-7B-instruct-v0.1 Mistral-7B-instruct-v0.2