Large Language Models versus Classical Machine Learning: Performance in COVID-19 Mortality Prediction Using High-Dimensional Tabular Data
<|reference_start|>Large Language Models versus Classical Machine Learning: Performance in COVID-19 Mortality Prediction Using High-Dimensional Tabular Data: Background: This study aimed to evaluate and compare the performance of classical machine learning models (CMLs) and large language models (LLMs) in predicting mortality associated with COVID-19 by utilizing a high-dimensional tabular dataset.   Materials and Methods: We analyzed data from 9,134 COVID-19 patients collected across four hospitals. Seven CML models, including XGBoost and random forest (RF), were trained and evaluated. The structured data was converted into text for zero-shot classification by eight LLMs, including GPT-4 and Mistral-7b. Additionally, Mistral-7b was fine-tuned using the QLoRA approach to enhance its predictive capabilities.   Results: Among the CML models, XGBoost and RF achieved the highest accuracy, with F1 scores of 0.87 for internal validation and 0.83 for external validation. In the LLM category, GPT-4 was the top performer with an F1 score of 0.43. Fine-tuning Mistral-7b significantly improved its recall from 1% to 79%, resulting in an F1 score of 0.74, which was stable during external validation.   Conclusion: While LLMs show moderate performance in zero-shot classification, fine-tuning can significantly enhance their effectiveness, potentially aligning them closer to CML models. However, CMLs still outperform LLMs in high-dimensional tabular data tasks.<|reference_end|>
@article{ghaffarzadeh-esfahani2024large,
    title={Large Language Models versus Classical Machine Learning: Performance in
  COVID-19 Mortality Prediction Using High-Dimensional Tabular Data},
    author={Mohammadreza Ghaffarzadeh-Esfahani, Mahdi Ghaffarzadeh-Esfahani, Arian
  Salahi-Niri, Hossein Toreyhi, Zahra Atf, Amirali Mohsenzadeh-Kermani, Mahshad
  Sarikhani, Zohreh Tajabadi, Fatemeh Shojaeian, Mohammad Hassan Bagheri, Aydin
  Feyzi, Mohammadamin Tarighatpayma, Narges Gazmeh, Fateme Heydari, Hossein
  Afshar, Amirreza Allahgholipour, Farid Alimardani, Ameneh Salehi, Naghmeh
  Asadimanesh, Mohammad Amin Khalafi, Hadis Shabanipour, Ali Moradi, Sajjad
  Hossein Zadeh, Omid Yazdani, Romina Esbati, Moozhan Maleki, Danial Samiei
  Nasr, Amirali Soheili, Hossein Majlesi, Saba Shahsavan, Alireza Soheilipour,
  Nooshin Goudarzi, Erfan Taherifard, Hamidreza Hatamabadi, Jamil S Samaan,
  Thomas Savage, Ankit Sakhuja, Ali Soroush, Girish Nadkarni, Ilad Alavi
  Darazam, Mohamad Amin Pourhoseingholi, Seyed Amir Ahmad Safavi-Naini},
    journal={arXiv preprint arXiv:2409.02136},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.02136},
    primaryClass={cs.LG cs.AI cs.CL}
}