Sara Whittaker, Natasha K Brusco, Keith D Hill, Nicholas F Taylor
Self-management programs within rehabilitation yield positive health outcomes at a small increased cost compared to usual care: A systematic review and meta-analysis
Archives of physical medicine and rehabilitation
national board osteopathic medical examiners nbome administers comprehensive osteopathic medical licensing examination united states comlexusa threelevel examination designed licensure practice osteopathic medicine examination design comlexusa level 3 l3 changed september 2018 twoday computerbased examination two components multiplechoice question mcq component single best answer clinical decisionmaking cdm case component extended multiplechoice emc short answer sa questions continued validation l3 examination especially new design essential appropriate interpretation use test scores purpose study gather evidence support validity l3 examination scores new design utilizing sources evidence based kanes validity framework kanes validity framework contains four components evidence support validity argument scoring generalization extrapolation implicationdecision study gathered data various sources conducted analyses provide evidence l3 examination validly measuring supposed measure include reviewing content coverage l3 examination documenting scoring reporting processes estimating reliability decision accuracyconsistency scores quantifying associations scores mcq cdm components scores different competency domains l3 examination exploring relationships l3 scores scores performancebased assessment measures related constructs performing subgroup comparisons describing justifying criterionreferenced standard setting process analysis data contains firstattempt test scores 8366 candidates took l3 examination september 2018 december 2019 performancebased assessment utilized criterion measure study comlexusa level 2 performance evaluation l2pe assessment forms built automated test assembly ata procedure maximize parallelism terms content coverage statistical properties across forms scoring reporting follows industrystandard qualitycontrol procedures interrater reliability sa rating decision accuracy decision consistency passfail classifications high statistically significant positive association mcq cdm components l3 examination patterns associations within l3 subscores l2pe domain scores fit measured subgroup comparisons gender race first language showed expected small differences mean scores subgroups within category yielded findings consistent described literature l3 passfail standard established implementation defensible criterionreferenced procedure study provides additional validity evidence l3 examination based kanes validity framework validity measurement must established ongoing evaluation related evidence nbome continue collect evidence support validity arguments comlexusa examination series