Table B2 Quantifying the Robustness of Inferences from Observational Studies in EEPA posted on-line July 24-Nov 15 2012 Study Random Predictor of Condition Outcome source r % bias to Multiplier (1st author) sample interest on pretest make to make (df)a inference estimate invalid significant Academic Effects of Summer RD 5th English & Table 2, .029 60% Summer Instruction and instruction and graders in Language Panel 1, Retention in New York retention New Arts column City York achievement 1 (Mariano) (29,987) 6th grade Unintended Consequences schools that 9th Peer ability Table 2, .030 50% of an Algebra-for-All increased Time graders in levels for high column Policy on High-Skill algebra Series Chicago skill students 1 Students enrollment (17,987) (1999 (Nomi) for difference for students affected by more than schools) 5%) Impact of Dual Enrollment yes Dual enrollment NELS Any degree Table 1, .043 50% on College Degree (8,754) obtained Panel A, Attainment column (An) 1 Life After Vouchers Student No 3rd-10th Math Table 5, .092 36% (Carlson) transferred to grade in all public school students public schools
Table B2 Quantifying the Robustness of Inferences from Observational Studies in EEPA posted on-line July 24-Nov 15 2012 Study (1st author) Random sample Predictor of interest Condition on pretest (df)a Outcome source r % bias to make inference invalid Multiplier to make estimate significant Academic Effects of Summer Instruction and Retention in New York City (Mariano) Summer instruction and retention RD 5th graders in New York (29,987) English & Language Arts achievement 6th grade Table 2, Panel 1, column 1 .029 60% Unintended Consequences of an Algebra-for-All Policy on High-Skill Students (Nomi) schools that increased algebra enrollment for students by more than 5%) Time Series 9th graders in Chicago (17,987) Peer ability levels for high skill students (1999 difference for affected schools) Table 2, column 1 .030 50% Impact of Dual Enrollment on College Degree Attainment (An) yes Dual enrollment NELS (8,754) Any degree obtained Table 1, Panel A, column 1 .043 50% Life After Vouchers (Carlson) Student transferred to public school No 3rd-10th grade in public schools Math Table 5, all students .092 36%
What%20would%20it%20take%20to%20Change%20an%20Inference%20published.docx