Lisa Z Scheifele, Nikolaos Tsotakos, Michael J Wolyniak
Preprint articles as a tool for teaching data analysis and scientific communication
PloS one
study assessed chatgpts ability generate readable accurate clear layperson summaries urological studies compared performance chatgptgenerated summaries original abstracts authorwritten patient summaries determine effectiveness potential solution creating accessible medical literature public articles top 5 ranked urology journals selected chatgpt prompt developed following guidelines maximize readability accuracy clarity minimizing variability readability scores gradelevel indicators calculated chatgpt summaries original abstracts patient summaries two md physicians independently rated accuracy clarity chatgptgenerated layperson summaries statistical analyses conducted compare readability scores cohens coefficient used assess interrater reliability correctness clarity evaluations total 256 journal articles included chatgptgenerated summaries created average time 175 sd 150 seconds readability scores chatgptgenerated summaries significantly better original abstracts global readability score 548 123 vs 298 185 flesch kincade reading ease 548 123 vs 298 185 flesch kincaid grade level 104 22 vs 135 40 gunning fog score 129 26 vs 166 41 smog index 91 20 vs 120 30 coleman liau index 129 21 vs 149 37 automated readability index 111 25 vs 120 57 p 0001 except automated readability index p 037 correctness rate chatgpt outputs 85 across categories assessed interrater agreement cohens 2 independent physician reviewers ranging 076095 chatgpt create accurate summaries scientific abstracts patients wellcrafted prompts enhancing userfriendliness although summaries satisfactory expert verification necessary improved accuracy