Mariah C Maxwell, Julia J Snyder, Ryan D P Dunk, Jeremy D Sloane, Isabella Cannon, Jason R Wiles
Peer-led team learning in an undergraduate biology course: Impacts on recruitment, retention, and imposter phenomenon
BMC research notes
emergency physicians need broad range knowledge skills address critical medical traumatic environmental conditions artificial intelligence ai including large language models llms potential applications healthcare settings however performance llms emergency medicine remains unclear evaluate reliability information provided chatgpt llm given questions set japanese association acute medicine board certification examinations period 5 years 20182022 programmed answer twice statistical analysis used assess agreement two responses llm successfully answered 465 475 textbased questions achieving overall correct response rate 623 questions without images rate correct answers 659 questions images explained llm rate correct answers 520 annual rates correct answers questions without images ranged 563 788 accuracy better scenariobased questions 691 standalone questions 621 agreement two responses substantial kappa 070 factual error accounted 82 incorrectly answered questions llm performed satisfactorily emergency medicine board certification examination japanese without images however factual errors responses highlight need physician oversight using llms