title: TAGS: A Test-Time Generalist-Specialist Framework with
    Retrieval-Augmented Reasoning and Verification
summary: Recent advances such as Chain-of-Thought prompting have significantly
  improved large language models (LLMs) in zero-shot medical reasoning. However,
  prompting-based methods often remain shallow and unstable, while fine-tuned
  medical LLMs suffer from poor generalization under distribution shifts and
  limited adaptability to unseen clinical scenarios. To address these
  limitations, we present TAGS, a test-time framework that combines a broadly
  capable generalist with a domain-specific specialist to offer complementary
  perspectives without any model fine-tuning or parameter updates. To support
  this generalist-specialist reasoning process, we introduce two auxiliary
  modules: a hierarchical retrieval mechanism that provides multi-scale exemplars
  by selecting examples based on both semantic and rationale-level similarity,
  and a reliability scorer that evaluates reasoning consistency to guide final
  answer aggregation. TAGS achieves strong performance across nine MedQA
  benchmarks, boosting GPT-4o accuracy by 13.8%, DeepSeek-R1 by 16.8%, and
  improving a vanilla 7B model from 14.1% to 23.9%. These results surpass several
  fine-tuned medical LLMs, without any parameter updates. The code will be
  available at https://github.com/JianghaoWu/TAGS.
paper_authors._id: 68358855d34b8e5da4e82fe6
paper_authors.name: Jianghao Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T10:03:31.484Z
paper_authors._id: 66de7dcb21f901b049895e33
paper_authors.avatarUrl: /avatars/a8beb6ef83afae6e40fdcf3a1fb08a62.svg
paper_authors.fullname: Jianghao
paper_authors.type: user
paper_authors.user: JianghaoWu
paper_authors._id: 68358855d34b8e5da4e82fe7
paper_authors.name: Feilong Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:39:04.112Z
paper_authors._id: 67467d9bf7b7f6fffeee8f09
paper_authors.avatarUrl: /avatars/da5dd1acd79478540c32b34060710e1b.svg
paper_authors.fullname: tangfeilong
paper_authors.type: user
paper_authors.user: tfl01
paper_authors._id: 68358855d34b8e5da4e82fe8
paper_authors.name: Yulong Li
paper_authors._id: 68358855d34b8e5da4e82fe9
paper_authors.name: Ming Hu
paper_authors._id: 68358855d34b8e5da4e82fea
paper_authors.name: Haochen Xue
paper_authors._id: 68358855d34b8e5da4e82feb
paper_authors.name: Shoaib Jameel
paper_authors._id: 68358855d34b8e5da4e82fec
paper_authors.name: Yutong Xie
paper_authors._id: 68358855d34b8e5da4e82fed
paper_authors.name: Imran Razzak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:39:16.051Z
paper_authors._id: 65ade1c114d782df06d92798
paper_authors.avatarUrl: /avatars/f47dffe30dd2af2ae774309fea017efe.svg
paper_authors.fullname: Imran Razzak
paper_authors.type: user
paper_authors.user: imranrazzakunsw
paper_title: TAGS: A Test-Time Generalist-Specialist Framework with
    Retrieval-Augmented Reasoning and Verification
paper_summary: Recent advances such as Chain-of-Thought prompting have significantly
  improved large language models (LLMs) in zero-shot medical reasoning. However,
  prompting-based methods often remain shallow and unstable, while fine-tuned
  medical LLMs suffer from poor generalization under distribution shifts and
  limited adaptability to unseen clinical scenarios. To address these
  limitations, we present TAGS, a test-time framework that combines a broadly
  capable generalist with a domain-specific specialist to offer complementary
  perspectives without any model fine-tuning or parameter updates. To support
  this generalist-specialist reasoning process, we introduce two auxiliary
  modules: a hierarchical retrieval mechanism that provides multi-scale exemplars
  by selecting examples based on both semantic and rationale-level similarity,
  and a reliability scorer that evaluates reasoning consistency to guide final
  answer aggregation. TAGS achieves strong performance across nine MedQA
  benchmarks, boosting GPT-4o accuracy by 13.8%, DeepSeek-R1 by 16.8%, and
  improving a vanilla 7B model from 14.1% to 23.9%. These results surpass several
  fine-tuned medical LLMs, without any parameter updates. The code will be
  available at https://github.com/JianghaoWu/TAGS.
paper_ai_summary: TAGS, a test-time framework combining generalist and specialist models with hierarchical retrieval and reliability scoring, enhances medical LLM reasoning without fine-tuning.
paper_ai_keywords: Chain-of-Thought prompting large language models (LLMs) zero-shot medical reasoning fine-tuning parameter updates hierarchical retrieval mechanism reliability scorer MedQA benchmarks GPT-4o DeepSeek-R1