Designing the Evaluation: What Evaluation Questions Are Addressed in an Outcomes-based Approach5 In-session evidence of: Assessment Process Concepts Event Agenda Event Environment Tasks Skills and Issues (productivity, participation) (logistics, organization) What will learners do in What skills must the What must the learner How did the activities How did the facilities, understand the training to learner master to to and facilitator contribute technology and demonstrate evidence of demonstrate what they demonstrate what they to participants’ learning? equipment contribute to what they can do? can do? can do? participants’ learning in For example, provided the session? For example, in-session For example, specific For example, specific activities that appealed to evidence of performance acts, ability to do tasks patterns, values, diverse ways of learning. For example, offered – assessment of tasks. process skills. motivations concepts web-based options to – – and issues. paper-based exercises. Did participants Can participants Can participants At our next educational What aspects of the effectively demonstrate demonstrate mastery demonstrate their event, how could facility supported (did not how they applied their of the intended skills? understanding of the participants support each support) your new skills? concepts and issues? other’s learning more participation? Can they list the skills? effectively? Can they list them? What two words would Tools: Can they communicate What did you do to you use to describe the Questionnaires or those skills to others? Can they communicate contribute to the success facilities? structured interviews those concepts and issues Tools: of the educational event, with participants to others? If we conduct the next Paper or electronic other than participate? education event at this Participant reflections questionnaires, What are you learning facility, what could be Simulations including Paper or electronic from other participants? improved? and tests questionnaires, Direct observations Tools: and tests Focus groups, Video or audio tapes Questionnaires paper Questionnaires paper – – interviews Focus groups, or electronic or electronic interviews Participant reflections Participant reflections Participant reflections Participant reflections Personal learning logs Structured interviews Structured interviews Personal learning logs Case analysis group Focus group Case analysis
Designing the Evaluation: What Evaluation Questions Are Addressed in an Outcomes-based Approach5 In-session evidence of: Assessment Tasks Process Skills Concepts and Issues Event Agenda (productivity, participation) Event Environment (logistics, organization) What will learners do in the training to demonstrate evidence of what they can do? For example, in-session evidence of performance – assessment of tasks. What skills must the learner master to demonstrate what they can do? For example, specific acts, ability to do tasks – process skills. What must the learner understand to demonstrate what they can do? For example, specific patterns, values, motivations – concepts and issues. How did the activities and facilitator contribute to participants’ learning? For example, provided activities that appealed to diverse ways of learning. How did the facilities, technology and equipment contribute to participants’ learning in the session? For example, offered web-based options to paper-based exercises. Did participants effectively demonstrate how they applied their new skills? Tools: Questionnaires or structured interviews with participants Participant reflections Simulations Direct observations Video or audio tapes Can participants demonstrate mastery of the intended skills? Can they list the skills? Can they communicate those skills to others? Tools: Paper or electronic questionnaires, including and tests Focus groups, interviews Participant reflections Personal learning logs Case analysis Can participants demonstrate their understanding of the concepts and issues? Can they list them? Can they communicate those concepts and issues to others? Paper or electronic questionnaires, and tests Focus groups, interviews Participant reflections Personal learning logs Case analysis At our next educational event, how could participants support each other’s learning more effectively? What did you do to contribute to the success of the educational event, other than participate? What are you learning from other participants? Tools: Questionnaires – paper or electronic Participant reflections Structured interviews group What aspects of the facility supported (did not support) your participation? What two words would you use to describe the facilities? If we conduct the next education event at this facility, what could be improved? Questionnaires – paper or electronic Participant reflections Structured interviews Focus group
CNSC-Toolkit-Evaluating-Education-Events-revised-Oct-2017.docx