Replication Data for: Bridging the Grade Gap: Reducing Assessment Bias in a Multi-Grader Class
<|begin_of_text|><|start_header_id|>system<|end_header_id|> You are a research archivist expert in dataset metadata. Answer the question truthfully.  <|eot_id|><|start_header_id|>user<|end_header_id|> Here is the Description of a dataset. Description:'Many large survey courses rely on multiple professors or teaching assistants to judge student responses to open-ended questions. Even following best practices, students with similar levels of conceptual understanding can receive widely varying assessments from different graders. We detail how this can occur and argue that it is an example of differential item functioning (or interpersonal incomparability), where graders interpret the same possible grading range differently. Using both actual assessment data from a large survey course in Comparative Politics and simulation methods, we show that the bias can be corrected by a small number of “bridging” observations across graders. We conclude by offering best practices for fair assessment in large survey courses.These files should fully replicate the findings in 'Bridging the Grade Gap' . Please return only a json list of at most 3 Keywords corresponding to the Description:.<|eot_id|><|start_header_id|>assistant<|end_header_id|> {"Answer" : ["Bayesian Aldrich-McKelvey scaling"]} <|eot_id|>
 Here is the Description of a dataset. Description:'Many large survey courses rely on multiple professors or teaching assistants to judge student responses to open-ended questions. Even following best practices, students with similar levels of conceptual understanding can receive widely varying assessments from different graders. We detail how this can occur and argue that it is an example of differential item functioning (or interpersonal incomparability), where graders interpret the same possible grading range differently. Using both actual assessment data from a large survey course in Comparative Politics and simulation methods, we show that the bias can be corrected by a small number of “bridging” observations across graders. We conclude by offering best practices for fair assessment in large survey courses.These files should fully replicate the findings in 'Bridging the Grade Gap' . Please return only a json list of at most 3 Keywords corresponding to the Description:.
Many large survey courses rely on multiple professors or teaching assistants to judge student responses to open-ended questions. Even following best practices, students with similar levels of conceptual understanding can receive widely varying assessments from different graders. We detail how this can occur and argue that it is an example of differential item functioning (or interpersonal incomparability), where graders interpret the same possible grading range differently. Using both actual assessment data from a large survey course in Comparative Politics and simulation methods, we show that the bias can be corrected by a small number of “bridging” observations across graders. We conclude by offering best practices for fair assessment in large survey courses.These files should fully replicate the findings in 'Bridging the Grade Gap
Bayesian Aldrich-McKelvey scaling