B.1. Data used to address implementation research questions (example data included in italics for expository purposes) of data used to assess whether the element of the intervention was implemented as Frequency/sampling of data Party responsible for data Implementation element intended collection collection Adherence: How often were sessions e.g., All sessions offered are captured e.g., All sessions delivered are e.g., Program staff offered? How many were offered? in MIS captured in MIS Program staff Length (number of minutes) of Session length sampled once a week program sessions captured in MIS Adherence: What and how much was e.g., Daily attendance records e.g., Student attendance at all e.g., Program staff received? sessions is captured in MIS Adherence: What content was e.g., Number of topics covered e.g., Classroom observations e.g., Evaluation staff delivered to youth? captured on observation occurred twice a year spreadsheeta Adherence: Who delivered material to e.g., List of staff members hired and e.g., Data on all staff members are e.g., Program staff youth? trained to implement program available to program staff Background qualifications of staff members from staff applications Quality: Quality of staff-participant e.g., Observations of interaction e.g., Convenience sample of 10% of e.g., Evaluation staff interactions quality using protocol developed by classroom sessions were selected for evaluators observation Quality: Quality of youth engagement e.g., Observations of engagement e.g., Random sample of 5% of all e.g., Evaluation with program using the YPQA sessions were selected for observation Counterfactual: Experiences of e.g., Survey items on baseline and e.g., Pre- and post-intervention e.g., Evaluation comparison condition follow-up assessments Convenience sample of comparison Focus groups of comparison group group participants (once) members Context: TPP programming e.g., District website listing all TPP e.g., Ad hoc e.g., Evaluation staff available or offered to study programming Once per year Evaluation staff participants (both intervention and Interview with school district comparison) curriculum director
B.1. Data used to address implementation research questions (example data included in italics for expository purposes) Implementation element of data used to assess whether the element of the intervention was implemented as intended Frequency/sampling of data collection Party responsible for data collection Adherence: How often were sessions offered? How many were offered? e.g., All sessions offered are captured in MIS Length (number of minutes) of program sessions captured in MIS e.g., All sessions delivered are captured in MIS Session length sampled once a week e.g., Program staff Program staff Adherence: What and how much was received? e.g., Daily attendance records e.g., Student attendance at all sessions is captured in MIS e.g., Program staff Adherence: What content was delivered to youth? e.g., Number of topics covered captured on observation spreadsheeta e.g., Classroom observations occurred twice a year e.g., Evaluation staff Adherence: Who delivered material to youth? e.g., List of staff members hired and trained to implement program Background qualifications of staff members from staff applications e.g., Data on all staff members are available to program staff e.g., Program staff Quality: Quality of staff-participant interactions e.g., Observations of interaction quality using protocol developed by evaluators e.g., Convenience sample of 10% of classroom sessions were selected for observation e.g., Evaluation staff Quality: Quality of youth engagement with program e.g., Observations of engagement using the YPQA e.g., Random sample of 5% of all sessions were selected for observation e.g., Evaluation Counterfactual: Experiences of comparison condition e.g., Survey items on baseline and follow-up assessments Focus groups of comparison group members e.g., Pre- and post-intervention Convenience sample of comparison group participants (once) e.g., Evaluation Context: TPP programming available or offered to study participants (both intervention and comparison) e.g., District website listing all TPP programming Interview with school district curriculum director e.g., Ad hoc Once per year e.g., Evaluation staff Evaluation staff
tpp_rpt_table_shells.docx