Quality statements Rating Evidence Key quality areas 1–6 and criteria Evaluation purpose and scope 1 Purpose of evaluation The purpose of the evaluation is provided, including the overall purpose and primary users of the information The scope matches the evaluation resources; 2 Scope of evaluation methods are defined and roles of the team, AusAID management and others are set out. Overall comments Evaluation methodology 3 Assessment of The evaluation assesses the intervention logic Intervention logic or theory, or equivalent, including underlying assumptions and factors. The report assesses the clarity of initiative objectives 4 Appropriateness of The methodology includes justification of the the methodology and design of the evaluation and the techniques use of sources for data collection and analysis. Methods are linked to and appropriate for each evaluation question. Triangulation is sufficient. The sampling strategy is appropriate (where applicable) Limitations to the methodology and any constraints encountered are described Ethical issues such as privacy, anonymity and cultural appropriateness are described and addressed 5 Adequacy and use of The adequacy of M&E data/systems are M&E described. The evaluation makes use of the existing M&E data. Overall comments Findings, conclusions and recommendations 6 The context of the The context of the initiative is described initiative (including policy, development and institutional context) and its influence on performance is assessed. 7 Evaluation questions The report identifies appropriate evaluation and criteria questions and then answers them. Any ratings, if given, are justified. Where this is not done, explanations are provided. An appropriate balance is made between operational and strategic issues. 8 Credibility of evidence Findings flow logically from the data, showing and analysis a clear line of evidence. Gaps and limitations in the data are clearly explained. Any assumptions are made explicit. Conclusions, recommendations and lessons are substantiated by findings and analysis. The relative importance of findings is stated clearly. The overall position of the author is unambiguous In assessing outcomes and impacts, attribution and/or contribution to results are explained. Alternative views / factors are explored to explain the observed results
Key quality areas and criteria Quality statements Rating 1–6 Evidence Evaluation purpose and scope 1 Purpose of evaluation The purpose of the evaluation is provided, including the overall purpose and primary users of the information 2 Scope of evaluation The scope matches the evaluation resources; methods are defined and roles of the team, AusAID management and others are set out. Overall comments Evaluation methodology 3 Assessment of Intervention logic The evaluation assesses the intervention logic or theory, or equivalent, including underlying assumptions and factors. The report assesses the clarity of initiative objectives 4 Appropriateness of the methodology and use of sources The methodology includes justification of the design of the evaluation and the techniques for data collection and analysis. Methods are linked to and appropriate for each evaluation question. Triangulation is sufficient. The sampling strategy is appropriate (where applicable) Limitations to the methodology and any constraints encountered are described Ethical issues such as privacy, anonymity and cultural appropriateness are described and addressed 5 Adequacy and use of M&E The adequacy of M&E data/systems are described. The evaluation makes use of the existing M&E data. Overall comments Findings, conclusions and recommendations 6 The context of the initiative The context of the initiative is described (including policy, development and institutional context) and its influence on performance is assessed. 7 Evaluation questions and criteria The report identifies appropriate evaluation questions and then answers them. Any ratings, if given, are justified. Where this is not done, explanations are provided. An appropriate balance is made between operational and strategic issues. 8 Credibility of evidence and analysis Findings flow logically from the data, showing a clear line of evidence. Gaps and limitations in the data are clearly explained. Any assumptions are made explicit. Conclusions, recommendations and lessons are substantiated by findings and analysis. The relative importance of findings is stated clearly. The overall position of the author is unambiguous In assessing outcomes and impacts, attribution and/or contribution to results are explained. Alternative views / factors are explored to explain the observed results