Large Model driven Radiology Report Generation with Clinical Quality Reinforcement Learning
<|reference_start|>Large Model driven Radiology Report Generation with Clinical Quality Reinforcement Learning: Radiology report generation (RRG) has attracted significant attention due to its potential to reduce the workload of radiologists. Current RRG approaches are still unsatisfactory against clinical standards. This paper introduces a novel RRG method, \textbf{LM-RRG}, that integrates large models (LMs) with clinical quality reinforcement learning to generate accurate and comprehensive chest X-ray radiology reports. Our method first designs a large language model driven feature extractor to analyze and interpret different regions of the chest X-ray image, emphasizing specific regions with medical significance. Next, based on the large model's decoder, we develop a multimodal report generator that leverages multimodal prompts from visual features and textual instruction to produce the radiology report in an auto-regressive way. Finally, to better reflect the clinical significant and insignificant errors that radiologists would normally assign in the report, we introduce a novel clinical quality reinforcement learning strategy. It utilizes the radiology report clinical quality (RadCliQ) metric as a reward function in the learning process. Extensive experiments on the MIMIC-CXR and IU-Xray datasets demonstrate the superiority of our method over the state of the art.<|reference_end|>
@article{zhou2024large,
    title={Large Model driven Radiology Report Generation with Clinical Quality
  Reinforcement Learning},
    author={Zijian Zhou, Miaojing Shi, Meng Wei, Oluwatosin Alabi, Zijie Yue, Tom
  Vercauteren},
    journal={arXiv preprint arXiv:2403.06728},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.06728},
    primaryClass={cs.CV}
}