Table 1 Quantifying the Robustness of Inferences from Randomized Experiments Study Treatment vs Blocking Population Outcome Estimated effect, Effect size % bias to make (author, year) Control standard error, (correlation) the inference source invalid A multi-site cluster Open Court Within 917 students Terra Nova 7.95 .16 47% randomized field trial curriculum grade and in 49 comprehensive (1.83) (.54) of Open Court versus school classrooms reading score Table 4, results Reading business as for reading (Borman et al., 2008) usual composite score Answers and questions Small classes By school 6500 Stanford 13.14 .23 64% about class size: A versus all students in Achievement (2.34) (.30) statewide experiment others 328 Test, reading Table 5 (Finn & Achilles, classrooms mean for other 1990) classes is based on the regular and aide classes combined proportional to their sample sizes. Experimental Building By 276 children Change in Early 3.55 .5 31% Evaluation of the Blocks program within 35 Mathematics (1.16) (.60) Effects of a research to type classrooms Childhood Building Blocks Preschool practice randomly Assessment IRT vs comparison Mathematics curriculum sampled scale score group Table 6 (df Curriculum versus from (mean=50, of 19 used based (Clements & Sarama, alternate math volunteers std=10) on footnote b) 2008) intensive within curriculum income strata
Table 1 Quantifying the Robustness of Inferences from Randomized Experiments Study (author, year) Treatment vs Control Blocking Population Outcome Estimated effect, standard error, source Effect size (correlation) % bias to make the inference invalid A multi-site cluster randomized field trial of Open Court Reading (Borman et al., 2008) Open Court curriculum versus business as usual Within grade and school 917 students in 49 classrooms Terra Nova comprehensive reading score 7.95 (1.83) Table 4, results for reading composite score .16 (.54) 47% Answers and questions about class size: A statewide experiment (Finn & Achilles, 1990) Small classes versus all others By school 6500 students in 328 classrooms Stanford Achievement Test, reading 13.14 (2.34) Table 5 mean for other classes is based on the regular and aide classes combined proportional to their sample sizes. .23 (.30) 64% Experimental Evaluation of the Effects of a Preschool Mathematics Curriculum (Clements & Sarama, 2008) Building Blocks research to practice curriculum versus alternate math intensive curriculum By program type 276 children within 35 classrooms randomly sampled from volunteers within income strata Change in Early Mathematics Childhood Assessment IRT scale score (mean=50, std=10) 3.55 (1.16) Building Blocks vs comparison group Table 6 (df of 19 used based on footnote b) .5 (.60) 31%
What%20would%20it%20take%20to%20Change%20an%20Inference%20published.docx