Evaluation vs. Research Concept Evaluation Research Planning Framework for program evaluation Scientific method Engage stakeholders State hypothesis Describe the program Collect data Focus the evaluation design Analyze data Gather credible evidence Draw conclusions Justify conclusions Ensure use and share lessons learned Decision Stakeholder-controlled Investigator-controlled Making Collaborative Authoritative Standards Repeatability Validity Utility Internal (accuracy, precision) Feasibility External (generalizability) Propriety Accuracy Questions Values Facts Merit (quality) Descriptions Worth (value) Associations Significance (importance) Effects Design Incorporate changes & account for Isolate changes & control situation circumstances Narrow experimental influences Expand to see all domains of influence Ensure stability over time Encourage flexibility and improvement Minimize context dependence Maximize context sensitivity Treat contextual factors as confounding Treat contextual factors as essential Comparison groups are a necessity information Understand that comparison groups are optional Data Sources Sources Collection Multiple (triangulation) Limited number (accuracy) Sampling strategies critical Sampling strategies critical Concern for protecting subjects, Concern for protecting subjects organizations, communities Indicators/ Measures Indicators/ Measures Quantitative Mixed-methods Qualitative Analysis & Timing Timing Synthesis Ongoing One-time (end) Scope Scope Integrate all data Focus on specific variables Judgements Explicit Implicit Examine agreement on values Tries to be value-free State whose values are used Conclusions Attribution & contribution Attribution Establish time sequence Establish time sequence Demonstrate plausible mechanisms Demonstrate plausibility Account for alternative explanations Control for confounding Show similar effects in similar contexts Replicate findings Uses Feedback to stakeholders Disseminate to interested audiences Focus on intended uses by intended users Content and format varies to maximize Build capacity comprehension Disseminate to interested audiences Content and format varies to maximize comprehension Emphasis on full disclosure Requirement for balanced assessment * Adapted from: U.S. Department of Health and Human Services Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. Introduction to program evaluation for public health programs: A self-study guide. Atlanta, GA: Centers for Disease Control and Prevention, 2011. Retrieved from:
Evaluation vs. Research Concept Evaluation Research Planning Framework for program evaluation Engage stakeholders Describe the program Focus the evaluation design Gather credible evidence Justify conclusions Ensure use and share lessons learned Scientific method State hypothesis Collect data Analyze data Draw conclusions Decision Making Stakeholder-controlled Collaborative Investigator-controlled Authoritative Standards Repeatability Utility Feasibility Propriety Accuracy Validity Internal (accuracy, precision) External (generalizability) Questions Values Merit (quality) Worth (value) Significance (importance) Facts Descriptions Associations Effects Design Incorporate changes & account for circumstances Expand to see all domains of influence Encourage flexibility and improvement Maximize context sensitivity Treat contextual factors as essential information Understand that comparison groups are optional Isolate changes & control situation Narrow experimental influences Ensure stability over time Minimize context dependence Treat contextual factors as confounding Comparison groups are a necessity Data Collection Sources Multiple (triangulation) Sampling strategies critical Concern for protecting subjects, organizations, communities Indicators/ Measures Mixed-methods Sources Limited number (accuracy) Sampling strategies critical Concern for protecting subjects Indicators/ Measures Quantitative Qualitative Analysis & Synthesis Timing Ongoing Scope Integrate all data Timing One-time (end) Scope Focus on specific variables Judgements Explicit Examine agreement on values State whose values are used Implicit Tries to be value-free Conclusions Attribution & contribution Establish time sequence Demonstrate plausible mechanisms Account for alternative explanations Show similar effects in similar contexts Attribution Establish time sequence Demonstrate plausibility Control for confounding Replicate findings Uses Feedback to stakeholders Focus on intended uses by intended users Build capacity Disseminate to interested audiences Content and format varies to maximize comprehension Emphasis on full disclosure Requirement for balanced assessment Disseminate to interested audiences Content and format varies to maximize comprehension * Adapted from: U.S. Department of Health and Human Services Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. Introduction to program evaluation for public health programs: A self-study guide. Atlanta, GA: Centers for Disease Control and Prevention, 2011. Retrieved from:
Evaluation%20vs.%20Research.docx