Automated Clinical Data Extraction with Knowledge Conditioned LLMs
<|reference_start|>Automated Clinical Data Extraction with Knowledge Conditioned LLMs: The extraction of lung lesion information from clinical and medical imaging reports is crucial for research on and clinical care of lung-related diseases. Large language models (LLMs) can be effective at interpreting unstructured text in reports, but they often hallucinate due to a lack of domain-specific knowledge, leading to reduced accuracy and posing challenges for use in clinical settings. To address this, we propose a novel framework that aligns generated internal knowledge with external knowledge through in-context learning (ICL). Our framework employs a retriever to identify relevant units of internal or external knowledge and a grader to evaluate the truthfulness and helpfulness of the retrieved internal-knowledge rules, to align and update the knowledge bases. Our knowledge-conditioned approach also improves the accuracy and reliability of LLM outputs by addressing the extraction task in two stages: (i) lung lesion finding detection and primary structured field parsing, followed by (ii) further parsing of lesion description text into additional structured fields. Experiments with expert-curated test datasets demonstrate that this ICL approach can increase the F1 score for key fields (lesion size, margin and solidity) by an average of 12.9% over existing ICL methods.<|reference_end|>
@article{li2024automated,
    title={Automated Clinical Data Extraction with Knowledge Conditioned LLMs},
    author={Diya Li, Asim Kadav, Aijing Gao, Rui Li, Richard Bourgon},
    journal={arXiv preprint arXiv:2406.18027},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.18027},
    primaryClass={cs.CL cs.AI}
}