Continuous improvement — the art of using data to decide goals, identify the paths to get there, evaluate the resulting change, and make adjustments to the next cycle — is the key to improving instructional practice.

School district leaders often find they might have lots of information — spreadsheets and spreadsheets of it — but it only covers the end result, disconnected from the actual question of “What’s working and what’s not?”.

Professional learning expert Joellen Killion refers to this as a “Black Box” scenario in her book Assessing Impact. “Black Box” evaluation often focuses on student data — which is important — but the work in between is important too. When you’re looking for teacher action to change, you want to capture data about whether or not that action is happening. Without studying how practice is changing along the way, Killion argues, we can’t reasonably conclude what drove the outcomes we achieved.

Consider Killion’s contrast of the black box with the “Glass Box”:

In a glass box setting, formative data acts as a flashlight to illuminate what’s actually happening during the implementation. This serves two benefits: district leaders can see what’s happening to target support in real time — and the formative data connects the inputs with outcomes, building the case for professional learning’s value.

Lighting up the glass box

Not all data is equally useful for improvement work. Formative assessment instruments work best when they’re consistent, simple, and relevant to the measured outcome. Each method, or way of gathering data, can yield many different types of data, for example:

Classroom visits

Periodic observations to “take the temperature” of how instruction is going

• Example form: Walkthrough survey filled out by observer
• Example data point: “Did this teacher assist all groups during small group work time,” Yes/No

Coaching

One-on-one assistance from instructional coaches

• Example form: Coaching and support logs, teacher self-assessments
• Example data point: Number of hours spent in coaching aligned to a specific outcome goal

PD events

Participant responses to professional development events

• Example form: Attendance records, post-event feedback
• Example data point: Teacher responses to the statement “I feel prepared to use restorative teaching practices in my classroom” before and after

A practical example

The richer your data, the more questions you can answer.  Maybe high school achievement scores on state ELA assessments increase, but K-8 results dip slightly.

• You look into it, and one light begins to illuminate the box: all teachers attended 25 hours of ELA-supportive PD, 30% more than last year.
• You look deeper, and another light appears: high school teachers rated their training highly, while 60% of K-8 teachers strongly disagreed with the statement “I am prepared to apply this training to my classroom.”
• Finally, you notice that the high school teachers’ personal learning networks (PLNs) chose classroom application as a focus this semester, while the K-8 teachers received no such support.

By cross-indexing the data, you’ve uncovered a valuable strategic move: reinforce classroom application skills through learning community or PLN work, as well as through “sit and get” training.

The value of glass-box thinking

Ultimately, glass box thinking isn’t just about better performance — it’s about demonstrating the return on investment of your professional learning. Education resources are precious. Coaching can have a tremendous impact, but it is costly, time-consuming, and highly dependent on quality. More and more, professional teams must demonstrate the value of their efforts to stakeholders at all levels in order to advocate for PD funding and resources.

With the right kind of information, a consistent data system, and a willingness to spend time on the details, you can build momentum in your district toward a continuously-improving workforce of happy, supported teachers.

‍