Suzanne Yang, Edward P Mulvey, Bruno Falissard
Textual data in psychiatry: reasoning by analogy to quantitative principles
The Journal of nervous and mental disease
item difficulty plays crucial role assessing students understanding concept tested difficulty item needs carefully adjusted ensure achievement evaluations objectives therefore study aimed investigate whether repeated item development training medical school faculty improves accuracy predicting item difficulty multiplechoice questions faculty development program implemented enhance prediction items difficulty index ensure absence item defects maintain general principles item development interrater reliability predicted actual corrected item difficulty assessed training using either kappa index correlation coefficient depending characteristics data total 62 faculty members participated training predictions item difficulty compared analysis results 260 items taken 119 fourthyear medical students 2016 316 items taken 125 fourthyear medical students 2018 training significant agreement predicted actual item difficulty indices observed one medical subject cardiology k 0106 p 0021 however training significant agreement noted four subjects internal medicine k 0092 p 0015 cardiology k 0318 p 0021 neurology k 0400 p 0043 preventive medicine r 0577 p 0039 furthermore significant agreement observed predicted actual difficulty indices across subjects analyzing average difficulty items r 0144 p 0043 regarding actual difficulty index subject neurology exceeded desired difficulty range 045075 2016 2018 however subjects fell within range repeated item development training includes predicting items difficulty index enhance faculty members ability predict adjust item difficulty accurately ensure difficulty examination aligns intended purpose item development training beneficial studies faculty development necessary explore benefits comprehensively