title: RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language
    Models
summary: The recent emergence of Medical Large Vision Language Models (Med-LVLMs) has
  enhanced medical diagnosis. However, current Med-LVLMs frequently encounter
  factual issues, often generating responses that do not align with established
  medical facts. Retrieval-Augmented Generation (RAG), which utilizes external
  knowledge, can improve the factual accuracy of these models but introduces two
  major challenges. First, limited retrieved contexts might not cover all
  necessary information, while excessive retrieval can introduce irrelevant and
  inaccurate references, interfering with the model's generation. Second, in
  cases where the model originally responds correctly, applying RAG can lead to
  an over-reliance on retrieved contexts, resulting in incorrect answers. To
  address these issues, we propose RULE, which consists of two components. First,
  we introduce a provably effective strategy for controlling factuality risk
  through the calibrated selection of the number of retrieved contexts. Second,
  based on samples where over-reliance on retrieved contexts led to errors, we
  curate a preference dataset to fine-tune the model, balancing its dependence on
  inherent knowledge and retrieved contexts for generation. We demonstrate the
  effectiveness of RULE on three medical VQA datasets, achieving an average
  improvement of 20.8% in factual accuracy. We publicly release our benchmark and
  code in https://github.com/richard-peng-xia/RULE.
paper_authors._id: 668c8f229761c585a2d8e11d
paper_authors.name: Peng Xia
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-09T01:16:39.696Z
paper_authors._id: 643e9ee6f6bb3c31a26e7bc4
paper_authors.avatarUrl: /avatars/acfaa7d6a23dada24c86b954c3be116a.svg
paper_authors.fullname: Peng Xia
paper_authors.type: user
paper_authors.user: richardxp888
paper_authors._id: 668c8f229761c585a2d8e11e
paper_authors.name: Kangyu Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T08:19:31.864Z
paper_authors._id: 661af01a8ad066992403cc6f
paper_authors.avatarUrl: /avatars/b4680010940a84c23521aa4b043ccab4.svg
paper_authors.fullname: Kangyu Zhu
paper_authors.type: user
paper_authors.user: zky11235
paper_authors._id: 668c8f229761c585a2d8e11f
paper_authors.name: Haoran Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-09T09:27:10.729Z
paper_authors._id: 64e86d6e7ef2c72cbcdb27b7
paper_authors.avatarUrl: /avatars/1968d11f5cd77af68fbd119402409fc1.svg
paper_authors.fullname: Haoran Li
paper_authors.type: user
paper_authors.user: brian13
paper_authors._id: 668c8f229761c585a2d8e120
paper_authors.name: Hongtu Zhu
paper_authors._id: 668c8f229761c585a2d8e121
paper_authors.name: Yun Li
paper_authors._id: 668c8f229761c585a2d8e122
paper_authors.name: Gang Li
paper_authors._id: 668c8f229761c585a2d8e123
paper_authors.name: Linjun Zhang
paper_authors._id: 668c8f229761c585a2d8e124
paper_authors.name: Huaxiu Yao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T08:20:16.465Z
paper_authors._id: 65f667a455009c4ad9e6ac4c
paper_authors.avatarUrl: /avatars/ed10c2cf2ba3fde6d7da93f076961607.svg
paper_authors.fullname: Yao
paper_authors.type: user
paper_authors.user: Huaxiu
paper_title: RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language
    Models
paper_summary: The recent emergence of Medical Large Vision Language Models (Med-LVLMs) has
  enhanced medical diagnosis. However, current Med-LVLMs frequently encounter
  factual issues, often generating responses that do not align with established
  medical facts. Retrieval-Augmented Generation (RAG), which utilizes external
  knowledge, can improve the factual accuracy of these models but introduces two
  major challenges. First, limited retrieved contexts might not cover all
  necessary information, while excessive retrieval can introduce irrelevant and
  inaccurate references, interfering with the model's generation. Second, in
  cases where the model originally responds correctly, applying RAG can lead to
  an over-reliance on retrieved contexts, resulting in incorrect answers. To
  address these issues, we propose RULE, which consists of two components. First,
  we introduce a provably effective strategy for controlling factuality risk
  through the calibrated selection of the number of retrieved contexts. Second,
  based on samples where over-reliance on retrieved contexts led to errors, we
  curate a preference dataset to fine-tune the model, balancing its dependence on
  inherent knowledge and retrieved contexts for generation. We demonstrate the
  effectiveness of RULE on three medical VQA datasets, achieving an average
  improvement of 20.8% in factual accuracy. We publicly release our benchmark and
  code in https://github.com/richard-peng-xia/RULE.
paper_ai_keywords: Medical Large Vision Language Models (Med-LVLMs) Retrieval-Augmented Generations (RAG) factuality risk provably enabled strategies preference dataset medical VQA datasets