Fiona Callan, Louise Keating, Sarah Casserley-Feeney, Helen P French
ADvAnced PhysioTherapy in MuSculosKeletal Triage: Investigating prognostic factors, healthcare utilisation and clinical outcomes (ADAPT MSK) - a cohort study protocol
HRB open research
communication core competency medical professionals utmost importance patient safety although medical curricula emphasize communication training traditional formats real simulated patient interactions present psychological stress limited repetition recent emergence large language models llms generative pretrained transformer gpt offers opportunity overcome restrictions aim study explore feasibility gptdriven chatbot practice history taking one core competencies communication developed interactive chatbot interface using gpt35 specific prompt including chatbotoptimized illness script behavioral component following mixed methods approach invited medical students voluntarily practice history taking determine whether gpt provides suitable answers simulated patient conversations recorded analyzed using quantitative qualitative approaches analyzed extent questions answers aligned provided script well medical plausibility answers finally students filled chatbot usability questionnaire cuq total 28 students practiced chatbot mean age 234 sd 29 years recorded total 826 questionanswer pairs qaps median 275 qaps per conversation 947 n782 pertaining history taking questions explicitly covered script n502 603 gptprovided answers mostly based explicit script information n471 944 questions covered script n195 234 gpt answers used 564 n110 fictitious information regarding plausibility 842 979 860 qaps rated plausible 14 21 implausible answers gpt provided answers rated socially desirable leaving role identity ignoring script information illogical reasoning calculation error despite results cuq revealed overall positive user experience 77100 points data showed llms gpt provide simulated patient experience yield good user experience majority plausible answers analysis revealed gptprovided answers use either explicit script information based available information understood abductive reasoning although rare gptbased chatbot provides implausible information instances major tendency socially desirable instead medically plausible information