Nicholas Benson, John H Kranzler, Randy G Floyd
Exploratory and Confirmatory Factor Analysis of the Universal Nonverbal Intelligence Test-Second Edition: Testing Dimensionality and Invariance Across Age, Gender, Race, and Ethnicity
Assessment
introduction artificial intelligence ai models using large language models llms nonspecific domains gained attention innovative information processing ai advances essential regularly evaluate tools competency maintain high standards prevent errors biases avoid flawed reasoning misinformation could harm patients spread inaccuracies study aimed determine performance chat generative pretrained transformer chatgpt openai google bard bard orthopedic surgery assess performance based question types contrast performance different ais compare ai performance orthopedic residents methods administered chatgpt bard 757 orthopedic intraining examination oite questions excluding imagerelated questions ais answered 390 multiple choice questions categorized within 10 subspecialties basic science trauma sports medicine spine hip knee pediatrics oncology shoulder elbow hand food ankle three taxonomy classes recall interpretation application knowledge statistical analysis performed analyze number questions answered correctly ai model performance returned ai model within categorized question subspecialty designation performance ai model comparison results returned orthopedic residents classified respective postgraduate year pgy level results bard answered overall questions correctly 58 vs 54 p0001 chatgpt performed better sports medicine basic science worse hand surgery bard performed better basic science p005 ais performed better recall questions compared application knowledge p005 based previous data ranked 42nd96th percentile postgraduate year ones pgy1s 27th58th pgy2s 3rd29th pgy3s 1st21st pgy4s 1st17th pgy5s discussion chatgpt excelled sports medicine fell short hand surgery ais performed well basic science subspecialty performed poorly application knowledgebased taxonomy questions bard performed better chatgpt overall although ai reached secondyear pgy orthopedic resident level fell short passing american board orthopedic surgery abos strengths recallbased inquiries highlight potential orthopedic learning educational tool