Table B1 Quantifying the Robustness of Inferences from Randomized Experiments in EEPA posted on-line July 24-Nov 15 2012 Study Predictor of Population Outcome source r % bias to Multiplier (1st author) interest (df) make to make inference estimate invalid significant Incentive Pay Programs Do Not Pay-for 5th-8th grade Test Table 3, .250 33% Affect Teacher Motivation or performance teachers in POINT Reported Practices Nashville program (Yuan) (143) Effects of Two Scientific Inquiry Professional 4th-5th grade Inquiry Table 5 .277 15% Professional Development development Classrooms in used Model 1 Interventions on Teaching for inquiry in Los Angeles Practice science (70 schools) (Grigg) A Randomized Controlled Trial Time to Read 8-9 year old Future Table 5, .078 7% Evaluation of Time to Read, a (tutoring) children in aspiration column Volunteer Tutoring Program for Northern 1 8- to 9-Year-Olds Ireland (Miller) (734) Incentive Pay Programs Do Not Pay-for 5th-8th grade Extra Table 3, .077 -154% 2.53 Affect Teacher Motivation or performance teachers in hours POINT Reported Practices Nashville worked/ program (Yuan) (143) week a Degrees of freedom for calculating threshold value of r are defined at the level of the predictor of interest, subtracting the number of parameters estimated at that level. Calculations can be conducted from spreadsheet at . Studies listed in order of % robustness.
Table B1 Quantifying the Robustness of Inferences from Randomized Experiments in EEPA posted on-line July 24-Nov 15 2012 Study (1st author) Predictor of interest Population (df) Outcome source r % bias to make inference invalid Multiplier to make estimate significant Incentive Pay Programs Do Not Affect Teacher Motivation or Reported Practices (Yuan) Pay-for performance 5th-8th grade teachers in Nashville (143) Test Table 3, POINT program .250 33% Effects of Two Scientific Inquiry Professional Development Interventions on Teaching Practice (Grigg) Professional development for inquiry in science 4th-5th grade Classrooms in Los Angeles (70 schools) Inquiry used Table 5 Model 1 .277 15% A Randomized Controlled Trial Evaluation of Time to Read, a Volunteer Tutoring Program for 8- to 9-Year-Olds (Miller) Time to Read (tutoring) 8-9 year old children in Northern Ireland (734) Future aspiration Table 5, column 1 .078 7% Incentive Pay Programs Do Not Affect Teacher Motivation or Reported Practices (Yuan) Pay-for performance 5th-8th grade teachers in Nashville (143) Extra hours worked/ week Table 3, POINT program .077 -154% 2.53 a Degrees of freedom for calculating threshold value of r are defined at the level of the predictor of interest, subtracting the number of parameters estimated at that level. Calculations can be conducted from spreadsheet at . Studies listed in order of % robustness.
What%20would%20it%20take%20to%20Change%20an%20Inference%20published.docx