behaviors (such as the ability to research a topic or construct an organized response). A comparison of the advantages of three popular item types (selected response, constructed response, and performance/portfolio observational items) is presented in Table 1. Note that the table assumes the item types are well conceived, developed, and implemented. Table 1. Comparative Advantages of Item Types Selected Response Constructed Response Performance/Portfolio/ (Objective Items) (Subjective Items) Observational Samples a lot of curriculum Samples less curriculum Samples less curriculum in a short period of time than selected response than selected response Sampling of items; takes longer items; takes longer Curriculum examinee administration examinee administration time time Requires the development Fewer items are needed Fewer items are needed, of many items but the items are written Item to break out the Development components of the task Can sample the range of Should be written for higher Can range the levels of Bloom's Revised Taxonomy levels of rigor rigor, although some from Remembering to items should represent Rigor Evaluating; takes skill to higher-level demands write items at the higher levels of rigor Low to moderate complexity Can range from low to high Tasks should reflect complexity moderate to high levels Complexity of complexity Objective scoring—efficient Subjective Subjective with a scoring key requires the use of requires the use of Scoring rubrics/scoring papers and rubrics; students can scorer training participate in scoring Overuse of the selected Good-quality constructed Encourages the response item format can response items can examinees to encourage learner passivity; encourage examinees to demonstrate what they can encourage development demonstrate creativity, know and can do. of critical thinking skills organizational skills, topic Depending on the item Influence on when items align with higher development, critical content, can encourage Learning levels of rigor thinking skills the development of critical thinking, organizational skills, and creativity High internal consistency Reliability is typically lower Reliability is typically reliability is possible with the than with selected response than with selected inclusion of 20+ high-quality items due to scorer response items due to Reliability items differences, fewer number of scorer differences, fewer items number of items Adapted from: Linn & Gronlund (1995). Other item qualities must also be considered when selecting items for DDMs. A high-quality assessment should be comprised of items that are not too difficult or too easy and should be populated with items that discriminate appropriately between strong and weak performers. For more information on the characteristics of high-quality items, see to Appendix B.
behaviors (such as the ability to research a topic or construct an organized response). A comparison of the advantages of three popular item types (selected response, constructed response, and performance/portfolio observational items) is presented in Table 1. Note that the table assumes the item types are well conceived, developed, and implemented. Table 1. Comparative Advantages of Item Types Selected Response (Objective Items) Constructed Response (Subjective Items) Performance/Portfolio/ Observational Sampling of Curriculum Samples a lot of curriculum in a short period of time Samples less curriculum than selected response items; takes longer examinee administration time Samples less curriculum than selected response items; takes longer examinee administration time Item Development Requires the development of many items Fewer items are needed Fewer items are needed, but the items are written to break out the components of the task Rigor Can sample the range of Bloom's Revised Taxonomy from Remembering to Evaluating; takes skill to write items at the higher levels of rigor Should be written for higher levels of rigor Can range the levels of rigor, although some items should represent higher-level demands Complexity Low to moderate complexity Can range from low to high complexity Tasks should reflect moderate to high levels of complexity Scoring Objective scoring—efficient with a scoring key Subjective the use of rubrics/scoring papers and scorer training Subjective the use of rubrics; students can participate in scoring Influence on Learning Overuse of the selected response item format can encourage learner passivity; can encourage development of critical thinking skills when items align with higher levels of rigor Good-quality constructed response items can encourage examinees to demonstrate creativity, organizational skills, topic development, critical thinking skills Encourages the examinees to demonstrate what they know and can do. Depending on the item content, can encourage the development of critical thinking, organizational skills, and creativity Reliability High internal consistency reliability is possible with the inclusion of 20+ high-quality items Reliability is typically lower than with selected response items due to scorer differences, fewer number of items Reliability is typically than with selected response items due to scorer differences, fewer number of items Adapted from: Linn & Gronlund (1995). Other item qualities must also be considered when selecting items for DDMs. A high-quality assessment should be comprised of items that are not too difficult or too easy and should be populated with items that discriminate appropriately between strong and weak performers. For more information on the characteristics of high-quality items, see to Appendix B. requires requires
TechnicalGuide.docx