Maureen Donnelly
On parts and holes: the spatial structure of the human body
Studies in health technology and informatics
large language models chatgpt openai demonstrated potential various applications including medical education previous studies assessed chatgpts performance university professional settings however models potential context standardized admission tests remains unexplored study evaluated chatgpts performance standardized admission tests united kingdom including biomedical admissions test bmat test mathematics university admission tmua law national aptitude test lnat thinking skills assessment tsa understand potential innovative tool education test preparation recent public resources 20192022 used compile data set 509 questions bmat tmua lnat tsa covering diverse topics aptitude scientific knowledge applications mathematical thinking reasoning critical thinking problemsolving reading comprehension logical reasoning evaluation assessed chatgpts performance using legacy gpt35 model focusing multiplechoice questions consistency models performance analyzed based question difficulty proportion correct responses aggregating exams years comparison test scores papers exam using binomial distribution pairedsample 2tailed tests proportion correct responses significantly lower incorrect ones bmat section 2 p001 tmua paper 1 p001 paper 2 p001 significant differences observed bmat section 1 p2 tsa section 1 p7 lnat papers 1 2 section p3 chatgpt performed better bmat section 1 section 2 p047 maximum candidate ranking 73 compared minimum 1 tmua engaged questions limited accuracy performance difference papers p6 candidate rankings 10 lnat demonstrated moderate success especially paper 2s questions however student performance data unavailable tsa performance varied across years generally moderate results fluctuating candidate rankings similar trends observed easy moderate difficulty questions bmat section 1 p3 bmat section 2 p04 tmua paper 1 p001 tmua paper 2 p003 tsa section 1 p8 lnat papers 1 2 section p99 hard challenging ones bmat section 1 p7 bmat section 2 p001 tmua paper 1 p007 tmua paper 2 p001 tsa section 1 p3 lnat papers 1 2 section p2 chatgpt shows promise supplementary tool subject areas test formats assess aptitude problemsolving critical thinking reading comprehension however limitations areas scientific mathematical knowledge applications highlight need continuous development integration conventional learning strategies order fully harness potential