by the number of text messages sent Quality What was the quality of Observations of interaction Percentage of sessions with high participant interactions—for quality, using protocol quality interactions, calculated example, were facilitators developed by evaluators as the percentage of observed knowledgeable and was pacing interactions that the independent adequate evaluator scored as “high quality” What was the quality of Observations of engagement, Percentage of sessions with engagement with using the Youth Program moderate youth engagement, program—for example, did the Quality Assessment (YPQA) calculated as the percentage of youths actively engage with sessions in which the the material, and were the Ratings from facilitator independent evaluator scored participants satisfied with the fidelity logs youth engagement as programming engaged” or higher “moderately Engagement ratings from participant satisfaction Average parent engagement surveys rating, calculated as the average of the engagement scale scores (ranging from 1–5) across parent satisfaction surveys Context and counterfactual What other TPP programming District website listing all List of TPP programming, all was available or offered to available TPP programming TPP programming available to study participants and those in both treatment and comparison the comparison condition—for Interviews with school groups described on district example, what are the sexual websites will be listed district curriculum director education requirements in the target area and is programming Percentage receiving in-class Survey items on baseline and available through community follow-up assessments TPP programming, the data on settings the survey question on experiences of the counterfactual at follow-up should be presented as frequency counts and percentages What external events affected List of school closures Number of schools that were implementation—for example, closed as a result of district was there staffing turnover, Interviews with community turnaround initiatives (unrelated were there budget cuts to the representatives to the TPP programming), if any program, and were there any technology issues in the area List of organizational changes Percentage and total number of planned cohorts missed due to community issues, if any What were the unplanned Documentation of request to List of unplanned adaptations, adaptations to key program or approval by OAH to make to be provided if any occur
by the number of text messages sent Quality What was the quality of interactions—for example, were facilitators knowledgeable and was pacing adequate Observations of interaction quality, using protocol developed by evaluators Percentage of sessions with high quality interactions, calculated as the percentage of observed interactions that the independent evaluator scored as “high quality” What was the quality of participant engagement with program—for example, did the youths actively engage with the material, and were the participants satisfied with the programming Observations of engagement, using the Youth Program Quality Assessment (YPQA) Ratings from facilitator fidelity logs Engagement ratings from participant satisfaction surveys Percentage of sessions with moderate youth engagement, calculated as the percentage of sessions in which the independent evaluator scored youth engagement as “moderately engaged” or higher Average parent engagement rating, calculated as the average of the engagement scale scores (ranging from 1–5) across parent satisfaction surveys Context and counterfactual What other TPP programming was available or offered to study participants and those in the comparison condition—for example, what are the sexual education requirements in the target area and is programming available through community settings District website listing all available TPP programming Interviews with school district curriculum director Survey items on baseline and follow-up assessments List of TPP programming, all TPP programming available to both treatment and comparison groups described on district websites will be listed Percentage receiving in-class TPP programming, the data on the survey question on experiences of the counterfactual at follow-up should be presented as frequency counts and percentages What external events affected implementation—for example, was there staffing turnover, were there budget cuts to the program, and were there any technology issues in the area List of school closures Interviews with community representatives List of organizational changes Number of schools that were closed as a result of district turnaround initiatives (unrelated to the TPP programming), if any Percentage and total number of planned cohorts missed due to community issues, if any What were the unplanned adaptations to key program Documentation of request to or approval by OAH to make List of unplanned adaptations, to be provided if any occur
2017-tpp-annual-progress-report-guidance-508.docx