Did you get a chance to catch the live Q&A on designing and monitoring effective summer PD? Here’s 7 highlights from our conversation with McREL International and Lubbock Independent School District, exploring how research and practice can inform using data to hone summer PD, focus on meeting individual needs, and drive towards change in practice.
Ready for more? RSVP for August’s interactive workshop with Joellen Killion and get a chance to analyze your data and take home key next steps for your professional learning program. Reserve your spot now!
Whenever we’re talking about any skills or strategies, it really helps to ground us in the why. Often when we think about evaluation, it feels like evaluation with a capital E, tied to things like test scores and statistical analysis. One of our favorite researchers around here, Joellen Killion, offers the term “black box” evaluation to describe this kind of high-stakes approach. We tie our inputs to our outcomes, and student testing data is usually the most common outcome:
In the black box, we don’t really know what happens in between, and we use whether or not we achieve student testing results to determine if those inputs worked. One of Killion’s arguments is that this approach feeds into this natural skepticism that we have about the results of professional learning. And so instead, she puts forth an idea of a glass box evaluation:
Really, it’s simple: what’s in the middle matters. Collecting implementation data along the way can give us the confidence that our results are actually tied to the changes that we made. But it also gives us the information we need to make to make our programs better.
As we look at professional development, and as district administrators are trying to decide the impact, and what’s going to make effective change, we pulled out several different buckets that aligned with the evidence and the research.
If you’re popping into a Zoom call as an administrator, what are you looking for? How do you set up the learning environment to know that your students are going to be successful, your teachers are going to be successful? Are there any purchases that need to be made? And so once you’ve laid the foundation (left), you can start focusing on facilitating online learning (right):
The sudden shift to online learning makes progress monitoring even more essential. School leaders know what good classroom instruction looks like—but in a digital world? It looks a little different. In addition to the stress of operating in a crisis, even our most veteran teachers are build new skills. So monitoring the implementation of learning (and ensuring professional learning is responsive to emerging needs) becomes the only way to keep the whole district moving towards high-quality instruction.
"When we added in KickUp, we had taken some really good steps toward measuring the implementation of strategies from our professional learning. And then comes along COVID to turn everything upside down. But we still need to know about implementation—and so we've shifted to making sure we're building the right capacity within our adult stakeholders so that they're doing the right things with students."
Executive Director of Leadership and Professional Development, Lubbock Independent School District
“As we’re thinking about our summer PD and what our fall looks like, our first question is, “Are you okay?” We have to ask our parents, are you okay? We have to ask our kids, are you okay?,” says Jackson. “All of this — the learning, the facilitating collaboration, the instructional materials, assessment, setting expectations — that only works when people feel like they’re okay. Taking care of that social-emotional component has to be the foundation.”
Lubbock looked at how teachers were feeling about the different platforms and tools and also how they’d like continued support. The Career and Technical Education (CTE) group — people who were already pretty good with technology — did great with this transition. But there are also groups who are having a tougher time. Performing Arts in a virtual format is a whole different beast. Now Jackson and her team can refocus efforts to ensure that those outlier groups with different needs get the attention they need.
Teachers felt pretty confident with Google Classroom and Screencastify, so their success rate was higher. But Screencastify had the second-largest number of requests for additional support, which told Jackson and her team that people were interested in adopting more advanced uses.
During spring, Lubbock used virtual PLCs as the primary format for helping teachers get from March to the end of the year. Each week, all instructional staff were expected to participate in at least one virtual PLC with their content-specific team and virtual PLCs with their campus as well.
“In the beginning, they had to touch base in both the morning and the afternoon,” says Jackson, “And then as we got a little bit better, it went to just the morning. And then as we got a little bit better, it didn’t have to be every single day.”
"We knew that there were some people hitting the ground running, and there were some people who hit the ground and felt like the ground hit them. So we had to create ongoing supports as they were building their their capacity and their learning."
Executive Director of Leadership and Professional Development, Lubbock Independent School District
In general, the recipe for a good virtual post-PD feedback form is the same as it is for an in-person event. You’re still trying to get to multiple facets of Guskey’s five levels. But sometimes that data collection becomes challenging to disambiguate.
You can imagine that in an online environment, application questions become even more important because of how personalized instruction can be. To bring it back to Guskey, Level Two can become very dispersed in online learning. If Teacher A takes Course 3, “Did they or did they not gain the skills and the strategies from course three?” becomes a question that’s very unique to that specific course.
Typically, we’re trying to balance that specificity with gathering data that’s applicable across everyone, so we can draw conclusions and see trends. So if we ask these really specific questions about the knowledge from Course 3, then we no longer have that intuitive heat map. Instead, we can embed the application questions — assessing skills and the knowledge — within the courses, and triangulate that with our overview about the experiences.