RACER: An LLM-powered Methodology for Scalable Analysis of Semi-structured Mental Health Interviews
<|reference_start|>RACER: An LLM-powered Methodology for Scalable Analysis of Semi-structured Mental Health Interviews: Semi-structured interviews (SSIs) are a commonly employed data-collection method in healthcare research, offering in-depth qualitative insights into subject experiences. Despite their value, the manual analysis of SSIs is notoriously time-consuming and labor-intensive, in part due to the difficulty of extracting and categorizing emotional responses, and challenges in scaling human evaluation for large populations. In this study, we develop RACER, a Large Language Model (LLM) based expert-guided automated pipeline that efficiently converts raw interview transcripts into insightful domain-relevant themes and sub-themes. We used RACER to analyze SSIs conducted with 93 healthcare professionals and trainees to assess the broad personal and professional mental health impacts of the COVID-19 crisis. RACER achieves moderately high agreement with two human evaluators (72%), which approaches the human inter-rater agreement (77%). Interestingly, LLMs and humans struggle with similar content involving nuanced emotional, ambivalent/dialectical, and psychological statements. Our study highlights the opportunities and challenges in using LLMs to improve research efficiency and opens new avenues for scalable analysis of SSIs in healthcare research.<|reference_end|>
@article{singh2024racer:,
    title={RACER: An LLM-powered Methodology for Scalable Analysis of
  Semi-structured Mental Health Interviews},
    author={Satpreet Harcharan Singh, Kevin Jiang, Kanchan Bhasin, Ashutosh
  Sabharwal, Nidal Moukaddam, Ankit B Patel},
    journal={arXiv preprint arXiv:2402.02656},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.02656},
    primaryClass={cs.CL q-bio.QM}
}