Sabrina Mangal, Stephanie Niño de Rivera, Jacky Choi, Meghan Reading Turchioe, Natalie Benda, Marianne Sharko, Annie Myers, Parag Goyal, Lydia Dugdale, Ruth Masterson Creber
Returning study results to research participants: Data access, format, and sharing preferences
International journal of medical informatics
american society metabolic bariatric surgery asmbs textbook serves comprehensive resource bariatric surgery covering recent advancements clinical questions testing artificial intelligence ai engines using authoritative source ensures accurate uptodate information provides insight potential implications surgical education training determine quality compare different large language models llms ability respond textbook questions relating bariatric surgery remote prompts entered llms multiplechoice questions found asmbs textbook bariatric surgery second edition prompts queried 3 llms openais chatgpt4 microsofts bing googles bard generated responses assessed based overall accuracy number correct answers according subject matter number correct answers based question type statistical analysis performed determine number responses per llms per category correct two hundred questions used query ai models overall significant difference accuracy answers accuracy 830 chatgpt4 followed bard 760 bing 650 subgroup analysis revealed significant difference models performance question categories chatgpt4s demonstrating highest proportion correct answers questions related treatment surgical procedures 831 complications 917 also significant difference performance different question types chatgpt4 showing superior performance inclusionary questions bard bing unable answer certain questions whereas chatgpt4 left questions unanswered llms particularly chatgpt4 demonstrated promising accuracy answering clinical questions related bariatric surgery continued ai advancements research required elucidate potential applications llms training education