This guest post is written by Kim T. Rodriguez, MA, Project Manager of the Evaluation Team at the Meadows Center for Educational Risk, which leads the evaluation effort at the National Disability Center.
“Evaluation is not an endpoint for us. It’s part of the iterative process that drives our work,” said Dr. Stephanie Cawthon, Principal Investigator and Executive Director. “The Evaluation Team attends our Leadership Team and All Hands meetings — providing insights and observing in real time how we are getting our projects done. This guest post by Kim reflects the National Disability Center’s commitment to sharing knowledge and embedding evaluation at every point in the process.”
Moving from a research project planned out on paper to implementation in real life is always an adventure. On the Evaluation Team at the Meadows Center for Educational Risk, we engage our clients in collaborative evaluation from the outset of this adventure.
When we joined the National Disability Center for Student Success, our formative evaluation process started with creating the logic model.
Using the narrative from the center’s grant application to the Institute for Education Science (IES) as a guide, our Evaluation Team — including myself, Dr. Greg Roberts (who is also Director of Evaluation on the National Disability Center’s Leadership Team and a Co-Principal Investigator), and Adah Barenburg — set about mapping the project inputs, planned activities, outputs, and expected outcomes into a preliminary working draft of what would ultimately become the National Disability Center Logic Model.
The Initial Draft
The initial draft is akin to trying to jump inside someone else’s brain to understand the ins and outs of the planned scope of work.
We understood that our first draft wouldn’t capture all the pieces of such a complex center with many moving parts created with input from thoughtful individuals from various backgrounds. The point of the initial logic model draft is never to get everything right; it is just to have a starting point to work from rather than a blank page.
As with any project, some National Disability Center team members had a deep familiarity with logic models, while for others, the concept of a logic model was wholly new. Our first step was to introduce the logic model concept broadly and then provide additional reading resources for team members needing more information.
Building a Bridge
Following adult learning principles, I presented this information electronically before the initial meeting with the Leadership Team. During the meeting, we discussed the logic model via metaphor; if the center is a bridge we are building, the logic model can be considered the bridge design. A bridge is built in a place, so context needs to be considered. In a logic model, the “inputs” are the context – the pieces already in place. The bridge’s construction is like the National Disability Center’s “activities” (what it will do). Once the bridge is built, it provides something tangible – a way to get from A to B. Those are the “outputs” in the logic model – generally countable tangibles.
However, a bridge provides something more, perhaps a connection or a new way that creates commuting efficiency. These relate to the “outcomes” of the logic model, which is what the activities and outputs will result in (e.g., behavior or attitude changes, new skills, and knowledge).
A bridge is not done and dusted once built, though. By design, it must be inspected periodically and maintained. Revisions to a logic model happen in two main ways:
Through continuous improvement, envisioned via the “Launch, Engage, Build, Engage, Iterate” philosophy included in the National Disability Center’s grant application, its goals, and now in its logic model.
In response to any change in course the National Disability Center makes in reaction to the needs of the field.
Therefore, our bridge is not constructed with concrete, so to speak. Instead, we envision a sturdy bridge made of Lego™ pieces — most of which will remain, but some of which will be switched out with new ones as needed.
Honing Each Draft
Once an understanding of the pieces of the logic model was explained, I used guiding questions to walk through each part of the logic model. Questions like, “Do you see your work/your place in this logic model? Where does it fit? What is missing?”
The Leadership Team undertook this questioning process and replicated it with the entire center at an All Hands meeting. Before and between these meetings, our team met with Dr. Stephanie Cawthon, the National Disability Center’s Executive Director, to review updated drafts that captured the revisions raised during each meeting, ensuring that everyone would see themselves and their work reflected in the logic model.
Each iteration of the logic model got us closer, and revisions became finer and finer grain until we reached the point of word-smithing.
The final step is production. If the logic model is our guide, it must look good! When all the parts and wording of the logic model were ready, the National Disability Center communications team transformed it into something bulletin board-worthy.
A Living Document
The logic model is a tool we will be constantly referencing, as it outlines the way the National Disability Center aims to achieve its purpose: to support the successful enrollment, persistence, and completion of degrees by disabled students. It will help the Center stay on track and within scope.
However, the logic model is a living document that may be revised in response to the center’s direction. Our job as evaluators is to maintain close collaboration with the National Disability Center’s activities and leaders so that we are aware of when such a revision is warranted.
Let the adventure continue!