HANDOUT #4.1 Module 4.1: Establish a Customized Standard-Setting Approach Standard-setting is the process of developing “cut scores” and performance level descriptors that identify performance at established points along the score. The modified contrasting group (MCG) approach is an examinee-centered method in which panelist classify the performance of students based upon both their knowledge of the and abilities in students’ skills, knowledge, conjunction with those expectations articulated in the content standards 4.1 Workflow: Establish a Customized Standard-Setting Approach 4.1 Procedural Steps: Modified Contrasting Groups (MCG) Approach 4.1.1 Design 4.1.1.a Preparation Identify targeted content standards and associated skills that the selected assessment evaluates. Gather a group of student samples to be reviewed by a qualified panel of scorers. The student samples should represent a wide range of performance/achievement. Determine criteria characteristic of a performance or a display representative “proficient” of content mastery, and the the assessment or task. minimum requirements for “passing” Record these criteria in the portion of the Design Establish a Customized Standard-Setting Approach Framework. Given the guidelines, determine criteria characteristic of “proficient” “advanced” performance and performance indicates “below proficient” performance. “Advanced” content mastery that exceeds the desired value, while performance “below proficient” indicates a significant and unsatisfactory deficit in demonstrating content mastery. Record these criteria in the portion of the Design Framework. Assign a performance level to and state a rationale for each selected student sample. Record these in the Framework. 4.1.1.b Approach Set a date for and notify potential panelists of a preliminary meeting that will evaluate the student work samples and train panelists in scorer calibration. (Note: procedures for scorer calibration are covered in depth in Module 3.3) Since this process relies almost solely on human judgement, a diverse group is essential. Panelists should have experience with the subject matter/grade level being evaluated, as well as with students participating in this subject and grade level. Subject matter experts,
HANDOUT #4.1 Module 4.1: Establish a Customized Standard-Setting Approach Standard-setting is the process of developing “cut scores” and performance level descriptors that identify performance at established points along the score. The modified contrasting group (MCG) approach is an examinee-centered method in which panelist classify the performance of students based upon both their knowledge of the students’ skills, knowledge, and abilities in conjunction with those expectations articulated in the content standards 4.1 Workflow: Establish a Customized Standard-Setting Approach 4.1 Procedural Steps: Modified Contrasting Groups (MCG) Approach 4.1.1 Design 4.1.1.a Preparation Identify targeted content standards and associated skills that the selected assessment evaluates. Gather a group of student samples to be reviewed by a qualified panel of scorers. The student samples should represent a wide range of performance/achievement. Determine criteria characteristic of a “proficient” performance or a display representative of content mastery, and the minimum requirements for “passing” the assessment or task. Record these criteria in the Design portion of the Establish a Customized Standard-Setting Approach Framework. Given the “proficient” guidelines, determine criteria characteristic of “advanced” performance and “below proficient” performance. “Advanced” performance indicates content mastery that exceeds the desired value, while “below proficient” performance indicates a significant and unsatisfactory deficit in demonstrating content mastery. Record these criteria in the Design portion of the Framework. Assign a performance level to and state a rationale for each selected student sample. Record these in the Framework. 4.1.1.b Approach Set a date for and notify potential panelists of a preliminary meeting that will evaluate the student work samples and train panelists in scorer calibration. (Note: procedures for scorer calibration are covered in depth in Module 3.3) Since this process relies almost solely on human judgement, a diverse group is essential. Panelists should have experience with the subject matter/grade level being evaluated, as well as with students participating in this subject and grade level. Subject matter experts,