Actionable data: Logic models, teacher PD, and connecting activities to outcomes

Pencil
By Julia Ramsey

It’s never been more important for education systems to demonstrate impact from their professional learning programs. The policy shift from test-based outcomes to more flexible, comprehensive success standards has brought new possibilities—and challenges—to every level of operations, including professional learning.

Join KickUp as we examine six of the most common roadblocks to thriving data cultures. With practical tips, guiding questions, and real-life examples of districts excelling through data, you can push your professional learning program to new levels of effectiveness.


Most data issues can be traced back to either design or implementation. Let’s start at the very beginning: how you design data gathering.

 

TACKLING COMMON DATA ISSUES | PROBLEM 1

The data isn’t connected to the practice being changed

 

Data-driven improvement begins with clear, explicit alignment between what you plan to change (inputs) and the intended results of that change (outcomes). Education researcher Joellen Killion’s “black box evaluation versus glass box evaluation” concept offers a clear metaphor for understanding the importance of this alignment in any successful program evaluation.

Let’s say we have a district-wide goal of raising student achievement in the literacy domain by 15%. We hypothesize that teachers using Cross-Curricular Literacy Training (CCLT) will become more equipped to support students’ literacy performance, so we implement CCLT coursework during summer professional development and throughout the year’s PD sessions.

In a “black box” program evaluation, the process looks something like this:

Conceptual image based on Joellen Killion, Assessing Impact: Evaluating Staff Development

At the end of the year, the literacy evaluations scores have indeed risen — but not by enough to hit the district goal. It’s now time to plan for next year, but how do we know what to do differently?

Joellen Killion’s “Glass Box” concept underscores the importance of understanding what comes between the inputs and the impact. Formative data, she argues, is just as critical as summative. By studying the activities happening during the implementation of professional learning programs, we can not only capture leading indicators that tell us if we’re on the right path—we can understand the why behind our positive outcomes (or lack thereof).

Training on the district scale requires taking input from so many people and analyzing it... and then actually making the adjustments. If we didn't have the data, I don't think that we would be nearly as effective in what we're doing.

Pencil

Anna Jackson

Executive Director of Leadership and Professional Development, Lubbock Independent School District

A logic model is a graphical representation that outlines the relationships between the resources, activities, outcomes, and intended impact of your professional learning program. An effective logic model aligns the programming to the programming’s goals—which makes it the perfect outline for evaluation. 

But what does “alignment” actually mean in this context? Two key factors:

Creating a Logic Model

Logic models bring planning, foresight and action together into a visual model of your program’s operations. They’re typically constructed before a program begins, providing an overview of the available resources, planned activities, predicted outcomes, and ultimate impact.

A program logic model… provides a road map of your program, highlighting how it is expected to work, what activities need to come before others, and how desired outcomes are achieved.

Pencil

W.K. Kellogg Foundation

WKKF Evaluation Handbook

Logic model components can vary by organization, but usually break down into:

Planning components
The process elements under your control.

Outcome components
Intended effects of your program

A logic model, then, becomes the perfect backbone for identifying precisely what practices are being changed and what data would indicate that change. So in the context of the literacy programming above, the model may look something like:

As we’ll explore further, those gold arrows are perfect opportunities for data check-ups. Confirming that Inputs are on the right track for effective Activities may mean simply verifying that the correct trainings are chosen and scheduled. Between activities & outputs, you might collect post-PD feedback. Between outputs and outcomes, you do observations to ensure classroom practice is changing. Each piece of data is carefully connected to the precise change it indicates, and each of those changes is connected to the larger goal.

Now, as the program progresses throughout the year, the “black box” turns into a glass one — illuminated by the metrics and sources outlined in the logic model.


Get a sneak peek of what’s ahead

Are you working to close the knowing-doing gap in teacher practice? The next article will cover turning subjective goals to concrete outcomes, but don’t wait — download the full “Actionable Data” e-book now to dig into more research, practitioner tips, and district exemplars:

From the blog