This session provides a forum for discussing the challenges of evaluating program effectiveness by using existing measures that vary in their alignment with program learning goals.
This session provides a forum for discussion around issues of alignment between programs, their learning goals, and the measures for assessing program effectiveness. The session seeks to offer ideas and strategies on how to tackle these kinds of issues.
The session addresses a common challenge in research—the tension between the need for using existing measures to ensure that research results can be compared across studies and the need for using measures that are well-aligned with a program’s learning goals to assess treatment effects.
The session consists of three parts. In the first part, presenters introduce the Differentiated PD project (DPD), a blended program to help teachers improve their knowledge of fractions. The presenters describe the main learning goals of the program to provide a context for exploring the question of alignment. In the second part, presenters introduce one of the measures used to assess program effectiveness—the Classroom Video Analysis (CVA) assessment—which is designed to measure useable mathematics teaching knowledge. Participants complete one of the CVA items to start a discussion around alignment issues. In the third part of the session, presenters introduce the Evaluating DMI project, a professional development program designed to help teachers improve their content and student thinking, and examine program effects on teacher knowledge, teaching quality, and student learning. In addition, the presenters introduce a new measure of instructional quality, developed under the Assessing Instructional Quality (AIS) project on the topic of fractions as a third example to discuss issues of alignment; invite session participants to share issues around alignment from their own work; and brainstorm ideas and strategies to tackles these important issues.