Alignment to Insufficient evidence The measures are The measures The measures Standards not aligned to partially reflect the reflect the full depth targeted depth and breadth and breadth of standards. of targeted targeted grade-level standards. standards. Content Validity: Insufficient evidence The measure only The measure The measure instructional peripherally addresses content addresses content sensitivity addresses content taught primarily in explicitly taught in that can be taught in classes, though class; rarely if ever classes, while some students may is this material mostly addressing have learned the learned outside of developmental content outside of school. issues or skills school. acquired outside of school. Reliability: items Insufficient evidence The number of There are multiple There are sufficient aligned, but insufficient items items aligned to is clearly aligned to content content to enable insufficient for for reliable reliable reliability. measurement of measurement of outcomes. outcomes. Reliability: scoring Insufficient evidence There are no are general There are precise of scoring criteria scoring criteria that scoring criteria responses related to the are not specifically related to the performance related to the performance expectations. performance expectations. expectations. Reliability: Insufficient evidence There are no There are limited are clear training procedures for procedures for procedures for training raters of training raters of training raters of open-ended open-ended open-ended responses. responses. responses. Reliability of scores Insufficient evidence is no evidence There is evidence There is evidence of score or rater that the scores or that the scores or reliability across raters have low raters are similar students in reliability across reasonably reliable varied contexts. similar students in across students in varied contexts. varied contexts. Fairness and Insufficient evidence There are many There are some The items are free freedom from bias items that contain items that contain of elements that elements that would elements that would would prevent some prevent some prevent some subgroups of of students groups of students students from from showing their from showing their showing their capabilities. capabilities. capabilities. Adapted from the work of Dr. Margaret Heritage, Assessment and Accountability Comprehensive Center, CRESST, April, 2012 (May 22, 2012)
Alignment to Standards Insufficient evidence The measures are not aligned to targeted standards. The measures partially reflect the depth and breadth of targeted standards. The measures reflect the full depth and breadth of targeted grade-level standards. Content Validity: instructional sensitivity Insufficient evidence The measure only peripherally addresses content that can be taught in classes, while mostly addressing developmental issues or skills acquired outside of school. The measure addresses content taught primarily in classes, though some students may have learned the content outside of school. The measure addresses content explicitly taught in class; rarely if ever is this material learned outside of school. Reliability: items Insufficient evidence The number of aligned, is clearly insufficient for reliability. There are multiple but insufficient items aligned to content for reliable measurement of outcomes. There are sufficient items aligned to content to enable reliable measurement of outcomes. Reliability: scoring of responses Insufficient evidence There are no scoring criteria related to the performance expectations. are general scoring criteria that are not specifically related to the performance expectations. There are precise scoring criteria related to the performance expectations. Reliability: training Insufficient evidence There are no procedures for training raters of open-ended responses. There are limited procedures for training raters of open-ended responses. are clear procedures for training raters of open-ended responses. Reliability of scores Insufficient evidence is no evidence of score or rater reliability across similar students in varied contexts. There is evidence that the scores or raters have low reliability across similar students in varied contexts. There is evidence that the scores or raters are reasonably reliable across students in varied contexts. Fairness and freedom from bias Insufficient evidence There are many items that contain elements that would prevent some of students from showing their capabilities. There are some items that contain elements that would prevent some of students from showing their capabilities. The items are free of elements that would prevent some subgroups of students from showing their capabilities. Adapted from the work of Dr. Margaret Heritage, Assessment and Accountability Comprehensive Center, CRESST, April, 2012 (May 22, 2012) groups
PartVII.docx