Integrating Large Language Models into a Tri-Modal Architecture for Automated Depression Classification
<|reference_start|>Integrating Large Language Models into a Tri-Modal Architecture for Automated Depression Classification: Major Depressive Disorder (MDD) is a pervasive mental health condition that affects 300 million people worldwide. This work presents a novel, BiLSTM-based tri-modal model-level fusion architecture for the binary classification of depression from clinical interview recordings. The proposed architecture incorporates Mel Frequency Cepstral Coefficients, Facial Action Units, and uses a two-shot learning based GPT-4 model to process text data. This is the first work to incorporate large language models into a multi-modal architecture for this task. It achieves impressive results on the DAIC-WOZ AVEC 2016 Challenge cross-validation split and Leave-One-Subject-Out cross-validation split, surpassing all baseline models and multiple state-of-the-art models. In Leave-One-Subject-Out testing, it achieves an accuracy of 91.01%, an F1-Score of 85.95%, a precision of 80%, and a recall of 92.86%.<|reference_end|>
@article{patapati2024integrating,
    title={Integrating Large Language Models into a Tri-Modal Architecture for
  Automated Depression Classification on the DAIC-WOZ},
    author={Santosh V. Patapati},
    journal={arXiv preprint arXiv:2407.19340},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.19340},
    primaryClass={cs.CV cs.AI cs.LG cs.MM}
}