From Generalist to Specialist: Improving Large Language Models for Medical Physics Using ARCoT
<|reference_start|>From Generalist to Specialist: Improving Large Language Models for Medical Physics Using ARCoT: Large Language Models (LLMs) have achieved remarkable progress, yet their application in specialized fields, such as medical physics, remains challenging due to the need for domain-specific knowledge. This study introduces ARCoT (Adaptable Retrieval-based Chain of Thought), a framework designed to enhance the domain-specific accuracy of LLMs without requiring fine-tuning or extensive retraining. ARCoT integrates a retrieval mechanism to access relevant domain-specific information and employs step-back and chain-of-thought prompting techniques to guide the LLM's reasoning process, ensuring more accurate and context-aware responses. Benchmarking on a medical physics multiple-choice exam, our model outperformed standard LLMs and reported average human performance, demonstrating improvements of up to 68% and achieving a high score of 90%. This method reduces hallucinations and increases domain-specific performance. The versatility and model-agnostic nature of ARCoT make it easily adaptable to various domains, showcasing its significant potential for enhancing the accuracy and reliability of LLMs in specialized fields.<|reference_end|>
@article{grandinetti2024from,
    title={From Generalist to Specialist: Improving Large Language Models for
  Medical Physics Using ARCoT},
    author={Jace Grandinetti and Rafe McBeth},
    journal={arXiv preprint arXiv:2405.11040},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.11040},
    primaryClass={cs.CL physics.med-ph}
}