Theory of Action MadLibs:
Theory of Action MadLibs:

I confess. I love logic models. I nerd out on theories of action. I prize evaluation tools that can help articulate the simple, yet complex beauty of coherent design. I especially admire a theory of action statement that reads like poetry. But even the most elegant logic model is meaningless if it does not reflect true intentionality. How do we ensure that our evaluation is authentic?

As researchers invested in participatory research we believe that evaluation must engage the voices of program staff and educators to be truly actionable and authentic. After all, how can organizations use evaluation to improve their work if the people responsible for delivering results are not included in the process?

Having said that, evaluation is not always at the forefront of the minds of staff and instructors that do the on-the-ground work of a program. As my colleague Jacob Watson points out:

“Most youth-serving orgs I’ve known, program evaluation is viewed with reluctance and skepticism: it is seen as drudgery at best and distraction at worst. it often sits at the bottom of every meeting’s agenda, or is pushed off until funders come knocking, asking about outcomes for the next grant application cycle.”

On the surface, evaluation can seem dull. The discourse can be esoteric and inaccessible. The particularities of each metric — inputs, outputs, outcomes, SMART goals — can carry a rigidity that seems to evade the inspiring cadence of an organization’s culture and vision.

The truth is, however, that good participatory evaluation work can be meaningful, engaging, and even, dare we say, fun. This is precisely what we at Convergence Design Lab set out to do.

In December 2020, my colleagues, Margaret Conway and Jacob Watson from Convergence Design Lab, led a two hour virtual workshop with several teaching artists and program staff from our long-standing clients, Spy Hop.

This youth media organization came to us to help them build a set of measurable learning outcomes, a logic model and a theory of action for their Youth in Care programs.

Knowing that the workshop would be conducted virtually through Zoom, we challenged ourselves to deliver an engaging online experience. Drawing on our expertise in learning design best practices, we set out to accomplish the following:

Make learning visible, hands-on, tech-powered and production-centered. (These were media teaching artists after all.) Honor the voices, instincts, expertise and language of the practitioners throughout the workshop. Cultivate an environment of playfulness and experimentation that encouraged participants to try on and test out ideas in a safe and collaborative environment.

The end goal was to land on a success profile of their programs. This success profile process would be bolstered by evidence and examples of outcomes and inputs, and culminate in a set of micro Theory of Action Statements that could inform our next steps in designing an authentic evaluation plan.

These are the principles that guided our approach.

1. Make Learning Visible, Hands-On, Tech-Powered, and Production-Centered

To make learning visible, we used Mural to guide participants through a scaffolded sequence of actions using a shared digital whiteboard. The final Mural also serves as a living artifact that participants and facilitators can return to and use for further work. Convergence’s Director of Learning, Margaret Conway, spent an admirable amount of time ensuring that the Mural template would work flawlessly, and it did. In the before COVID times, we used paper and analogue means, but we were grateful to have a tool like Mural to simulate a design thinking workshop experience.

2. Honor the voices, instincts, expertise and language of the practitioners throughout the workshop.

During the workshop, participants were provided ample space and time to share reflections and ideas. We presented the first activity in the form of a challenge —build a success profile. Our prompts:

What is happening in the program when it is working?

What does it look like when things are humming along?

How do you know when students are getting it?’

The stories unearthed during this process were moving, rich and detailed. Participants easily shared success in their own words, shored up through personal experiences and everyday observations.

After capturing these Outcomes, we then introduced the concept of Inputs. How did these outcomes occur? What were the essential ingredients that created success?

The use of the word “ingredients” invited people to conceptualize the learning ecology as a signature dish comprised of many components working together to create a lasting and flavor-filled experience.

The creative and positive-thinking Spy Hop teaching artists needed no cajoling or hand-holding. They quickly populated the Mural board with digital post-its under pre-defined headings such as Mindsets, Process, Practices, Equipment.

Throughout the process, participants narrated their post-its, asked clarifying questions and explained their thinking. Sharing stories in this way presents an all-too-rare opportunity for practitioners to hear from one another as make new connections across their individual practices and experiences.

3. Cultivate an environment of playfulness and experimentation.

By the time we arrived at the final activity (creating theory of action statements), we worried that we might fall short on time. But as it turned out, the participants were primed for success during the lightning round.

We drew inspiration from MadLibs, the classic silly story-building activity, in order to build micro theories of action. MadLibs is low-stakes, and invites players to come up with words first, before putting them into any sort of order or story.

Drawing from word lists previously made — Success Stories, Outcomes and Inputs the teaching artists seamlessly used Mural’s copy/paste and drag and drop features to generate a flurry of theory of action mad libs using this simple template:

Spy Hop Theory of Action as Refrigerator Magnet Poetry

As each member populated the Mural board with their own content, we took note of the vivid phrases emerging before our eyes. “It’s like refrigerator poetry!”, someone said.

Screenshot from Mural Board used in Convergence Virtual Workshop

In the end, this playful and participatory evaluation design process not only made us more accountable as evaluators, but it helped strengthen Spy Hop’s community of practice. As one staff member wrote: “Being a new mentor, I learn a lot from meetings like this, and I am able to take this back into the classes that I am teaching.”

That is, afterall, what makes evaluation truly authentic.
['Mindy Faber']
['Evaluation', 'Participatory Design', 'Nonprofit', 'Education', 'Learning And Development']