Illuminating the PD process: 3 practical examples of “glass-box” data

Pencil
By Julia Ramsey

Continuous improvement — the art of using data to decide goals, identify the paths to get there, evaluate the resulting change, and make adjustments to the next cycle — is the key to improving instructional practice.

School district leaders often find they might have lots of information — spreadsheets and spreadsheets of it — but it only covers the end result, disconnected from the actual question of “What’s working and what’s not?”. 

Professional learning expert Joellen Killion refers to this as a “Black Box” scenario in her book Assessing Impact. “Black Box” evaluation often focuses on student data — which is important — but the work in between is important too. When you’re looking for teacher action to change, you want to capture data about whether or not that action is happening. Without studying how practice is changing along the way, Killion argues, we can’t reasonably conclude what drove the outcomes we achieved. 

Consider Killion’s contrast of the black box with the “Glass Box”:

In a glass box setting, formative data acts as a flashlight to illuminate what’s actually happening during the implementation. This serves two benefits: district leaders can see what’s happening to target support in real time — and the formative data connects the inputs with outcomes, building the case for professional learning’s value.

Lighting up the glass box

Not all data is equally useful for improvement work. Formative assessment instruments work best when they’re consistent, simple, and relevant to the measured outcome. Each method, or way of gathering data, can yield many different types of data, for example:

Classroom visits

Periodic observations to “take the temperature” of how instruction is going

Coaching

One-on-one assistance from instructional coaches

PD events

Participant responses to professional development events

Above: A real-life example from Comal ISD’s “Bridge to Practice” system — click here to see Hanna Schramm, Director of Professional Learning, explaining how Comal implemented it even during the challenges of COVID

A practical example

The richer your data, the more questions you can answer.  Maybe high school achievement scores on state ELA assessments increase, but K-8 results dip slightly. 

By cross-indexing the data, you’ve uncovered a valuable strategic move: reinforce classroom application skills through learning community or PLN work, as well as through “sit and get” training.

The value of glass-box thinking

Ultimately, glass box thinking isn’t just about better performance — it’s about demonstrating the return on investment of your professional learning. Education resources are precious. Coaching can have a tremendous impact, but it is costly, time-consuming, and highly dependent on quality. More and more, professional teams must demonstrate the value of their efforts to stakeholders at all levels in order to advocate for more of the increasingly-available funding and resources.

With the right kind of information, a consistent data system, and a willingness to spend time on the details, you can build momentum in your district toward a continuously-improving workforce of happy, supported teachers.

From the blog