Stephen R Ali|Thomas D. Dobbs|Hayley Hutchings|Iain S. Whitaker
Using ChatGPT to write patient clinic letters
Swansea University|Morriston Hospital|Swansea University|Morriston Hospital|Swansea University|Swansea University|Morriston Hospital
appropriate recording communication clinical information clinicians patients paramount importance recently muchneeded drive improve information shared patients1general medical councildomain 3 communication partnership teamworkhttpswwwgmcukorgethicalguidanceethicalguidancefordoctorsgoodmedicalpracticedomain3communicationpartnershipandteamworkparagraph31date accessed december 22 2022google scholar however preparation clinical letters time consuming although increase use letter templates voice recognition systems aim improving efficiency novel technologies natural language processing nlp artificial intelligence ai power revolutionise area practice nlp algorithms designed recognise understand structure meaning human language classify texts according content purpose generate responses appropriate coherent2jurafsky martin jh speech language processing3rd edn prentice hall 2009google scholar openais chatgpt chatbot launched november 2022 uses nlp technology generate humanlike text generative pretrained transformer gpt language model based transformer architecture allows process large amounts text data generate coherent text outputs learning relationships input output sequences gpt language model trained large datasets human language several studies demonstrating good generating highquality coherent text outputs3brown mann b ryder n et allanguage models fewshot learnersarxiv 2020 published online july 22 preprinthttpsdoiorg1048550arxiv200514165google scholar 4radford wu j child r et alexploring limits transfer learning unified texttotext transformerarxiv 2019 published online july 28 preprinthttpsdoiorg1048550arxiv191010683google scholar 5vaswani shazeer n parmar n et al attention need nips17 proceedings 31st international conference neural information processing systems december 2017 abstr 59986008google scholar result ai like chatgpt potential produce high quality clinical letters comprehendible patients improving efficiency consistency accuracy patient satisfaction deliver cost savings healthcare system comment describe early adoption evaluation chatgptgenerated clinical letters patients limited clinical input aim evaluate readability factual correctness humanness chatgptgenerated clinical letters patients using example skin cancer common human cancer created series different clinical communication scenarios covered remit clinicians skin cancer practice simulate clinicians might use chatgpt clinical environment created shorthand instructions input chatbot defined limited clinical input input small compared relative amount natural free text clinician would otherwise required write dictate generate clinical letter usa recommended patientfacing health literature written sixth grade level age 1112 years6us department health human servicesnational action plan improve health literacyhttpshealthgovsitesdefaultfiles201909healthliteracyactionplanpdfdate 2010date accessed december 22 2022google scholar specific guidelines uk view letters instructed written reading age 1112 years sought evaluate capabilities chatgpt presenting series instructions increasing complexity appendix p 1 instructions ranged simply following specific directions using national guidelines data guidelines provide clinical advice lettereg management anticoagulation perioperatively submitting instructions chatgpt generated response form clinical letter issued patient online tool readable used evaluate readability letters commonly used formulae described many studies7burke v greenberg determining readability select apply easytouse readability formulas assess difficulty adult literacy materialsadult basic educ lit 2010 4 3442google scholar 8wang lw miller mj schmitt mr wen fk assessing readability formula differences written health information materials application results recommendationsres social adm pharm 2013 9 503516crossref pubmed scopus 181 google scholar factual correctness humanness letters assessed two independent clinicians using likert scale ranging 0 10 0 representing completely incorrect inhuman 10 representing completely correct human error analysis performed using linear regression used two separate generalised linear models glms investigate effect predictor variables cancer type basal cell carcinoma bcc reference category general commands specific guidelines general guidelines outcome variables median humanness first glm median correctness second glm statistical analysis done r version 411 p0001 deemed statistically significant 38 hypothetical clinical scenarios created seven pertained bcc 11 squamous cell carcinoma scc 20 malignant melanoma mm appendix p 2 overall readability scores suggest text might suitable varying reading ability mean readability age generated letters usa ninth grade aged 1415 years considered us department health human services average difficulty appendix p 2 overall median correctness clinical information contained letter 7 range 19 overall median humanness writing style 7 59 appendix pp 34 weighted correctness 080 p00001 humanness 077 p00001 median correctness anova howed statistically significant difference among groups f235101 p000035 tukey honestly significant difference hsd test showed mean difference mm bcc groups statistically significant 271 95 ci 432 111 p00001 statistically significant difference scc bcc groups p031 scc mm groups p0016 median humanness anova showed statistically significant difference among groups f2352776 p00001 tukey hsd test showed statistically significant difference mm bcc groups 163 217 109 p00001 scc bcc groups 143 203 083 p00001 statistically significant difference scc mm groups p055 results glm median humanness showed cancer type significant predictor mm coefficient 145 se 030 scc coefficient 132 028 p00001 general commands specific guidelines general guidelines variables significant predictors median humanness multiple r2 value 0622 indicated model explained 6223 variance median humanness fstatistic 1055 corresponding p00001 indicated model significant overall glm median correctness mm found significantly associated median correctness coefficient 264 se 090 p00001 predictors significantly associated median correctness multiple r2 value 03729 suggested model explained approximately 3729 variance median correctness fstatistic corresponding p value used test overall significance model p00001 indicated model significantly different model predictors pilot assessment shows possible generate clinic letters high overall correctness humanness score chatgpt furthermore letters written reading level broadly similar current realworld human generated letters9drury dj kaur dobbs whitaker readability outpatient plastic surgery clinic letters adhering plain english writing standardsplast surg nurs 2021 41 2733crossref pubmed scopus 6 google scholar ability ai generate clinical letters alternative written clinicians raises important considerations quality effectiveness healthcare communication important potential risks omissions errors might serious consequences patient care mitigated incorrect reporting results interpretation treatment guidelines could affect patient morbidity mortality mitigate risks important use ai health care including automated generation clinical letters carefully regulated monitored early stages adoption new technologies necessary continue humanintheloop approach whereby outputs systems carefully verified healthcare providers responsibly incorporate chatgpt similar generic ai systems clinical workflow one approach could use voicetotext recognition software limited human input followed rapid clinician editing generated letter approach could feasible starting point exploring potential applications technology also addressing potential risks studies needed assess effectiveness aigenerated clinical letters realworld clinical settings compare performance various ai systems one another probable ai system greater medical focus yield improved results additionally quality writing style input provided chatbot assessed range settings including different languages resource levels cultural contexts caution must exercised potential risks must proactively addressed ensure safety quality patient care introducing important technical advancement declare competing interests sra tdd funded welsh clinical academic training fellowship sra received paton masser grant british association plastic reconstructive aesthetic surgeons support work isw surgical specialty lead health care research wales chief investigator scar free foundation health care research wales programme reconstructive regenerative surgery research associate editor annals plastic surgery editorial board member bmc medicine numerous editorial board roles ethical committee approval obtained swansea university medical school research ethics subcommittee 20200025 study performed accordance declaration helsinki download pdf 43 mb help pdf files supplementary appendix