Steve Strand
Consistency in reasoning test scores over time
The British journal of educational psychology
large language models chatgpt gained popularity ability generate comprehensive responses human queries field medicine chatgpt shown promise applications ranging diagnostics decisionmaking however performance medical examinations comparison random guessing extensively studied study aimed evaluate performance chatgpt preinternship examination comprehensive medical assessment students iran examination consisted 200 multiplechoice questions categorised basic science evaluation diagnosis decisionmaking gpt4 used questions translated english statistical analysis conducted assess performance chatgpt also compare random test group results showed chatgpt performed exceptionally well 685 questions answered correctly significantly surpassing pass mark 45 exhibited superior performance decisionmaking successfully passed specialties comparing chatgpt random test group chatgpts performance significantly higher demonstrating ability provide accurate responses reasoning study highlights potential chatgpt medical licensing examinations advantage random guessing however important note chatgpt still falls short human physicians terms diagnostic accuracy decisionmaking capabilities caution exercised using chatgpt results verified human experts ensure patient safety avoid potential errors medical field