AERA 2017 Poster: Evaluating the Developing Mathematical Ideas Professional Development Program

AERA Poster

Click image to see poster.

Abstract

Objectives: This NSF-funded (DRL-1019769) experimental and quasi-experimental efficacy study of teacher participation in the Developing Mathematical Ideas (DMI) elementary math professional development (PD) program (Schifter, Bastable, & Russell, 1999-2008), focuses on its impact on teacher knowledge, teaching practice, and student achievement. Working with teachers and students in MA and NC schools over several years, the project uses existing measures and builds new measures of teacher content knowledge and teaching practices, and explores elementary student engagement with Common Core Math Practices.

Perspectives: The logic of the intervention is that by deepening teachers’ understanding of mathematical ideas and how students think and learn about those ideas, teachers will change their teaching practice, especially to emphasize increased discourse about mathematical ideas, which will, in turn, improve student understanding and ability to justify mathematical ideas.

Methods: Grades 1-5 teachers from eight districts in Massachusetts and North Carolina volunteered to be randomly assigned to be offered number and operations-focused Building a System of Tens (BST) and Making Meaning for Operations (MMO) intensive math PD in the first year of their participation, or to wait a year and serve as the comparison group. Several assessments of teacher knowledge, rubrics applied to video of classroom teaching, and measures of student achievement were collected and analyzed from all teachers across two years. Analyses look for impacts in each area, and links among them. The project also analyzes classroom video for evidence of students engaging in Common Core Math Practices (Common Core State Standards Initiative, 2010).

Data Sources: Teacher knowledge measures, administered annually, included the LMT Math Knowledge for Teaching (MKT) measure (Ball, Hill, Rowan, & Schilling, 2002; Hill, Schilling, & Ball, 2004); an open-ended assessment aligned with DMI content (OE) (Higgins, Bell, Wilson, Oh, & McCoach, 2007); and a new Classroom Video Analysis (CVA) measure (Kersting, 2008) focusing on number and operations concepts. Classrooms were videotaped annually and videos were scored for teaching practices using the Mathematical Quality of Instruction (MQI) measure (Hill & Learning Mathematics for Teaching, 2014) as well as using three new rubrics we developed in the style of the MQI addressing the extent to which students engage in detail with one another’s mathematical ideas and how teachers support that (Franke et al., 2007; Webb et al., 2013). Student achievement was measured using end-of-unit and end-of-year assessments developed by the Boston Public Schools and aligned with the Investigations curriculum (TERC, 2008), which was used across all study sites to reduce confounding teaching practice with curriculum-in-use.

Results: Results suggest that DMI PD leads to gains in teacher knowledge – for Treatment group teachers in Year 1, and Comparison group teachers in Year 2, each when they take DMI. Teaching practice results are mixed and complicated, but suggest that changes may not occur until the year after PD participation. We find no statistically significant differences in student achievement by Treatment group.

Significance: This study of high quality DMI math PD provides evidence of its impact and possible mechanisms for creating change that can inform policy and practice.

References:
Ball, D. L., Hill, H. C., Rowan, B., & Schilling, S. G. (2002). Measuring teachers' content knowledge for teaching: Elementary mathematics release items, 2002. Retrieved from Ann Arbor, MI.
Common Core State Standards Initiative. (2010). Common Core State Standards for Mathematics. Washington, DC: Authors.
Franke, M. L., Webb, N. M., Chan, A., Battey, D., Ing, M., Freund, D., & De, T. (2007). Eliciting student thinking in elementary school mathematics classrooms (725). Retrieved from Los Angeles, CA.
Higgins, T. L., Bell, C. A., Wilson, S. M., Oh, Y., & McCoach, D. B. (2007). Measuring the impact of professional development. Paper presented at the The NCTM Research Presession, Atlanta, GA.
Hill, H. C., & Learning Mathematics for Teaching. (2014). Mathematical Quality of Instruction (MQI) 4-point version. Teaching Practice Observation Rubrics. Cambridge, MA.
Hill, H. C., Schilling, S. G., & Ball, D. L. (2004). Developing measures of teachers' mathematics knowledge for teaching. The Elementary School Journal, 105(1), 11-30.
Kersting, N. (2008). Using video clips as item prompts to measure teachers' knowledge of teaching mathematics. Educational & Psychological Measurement, 68(845-861).
Schifter, D., Bastable, V., & Russell, S. J. (1999-2008). Developing Mathematical Ideas: Casebooks & facilitator's guides. Parsippany, NJ: Dale Seymour Publications.
TERC. (2008). Investigations in Number, Data and Space, Grades K–5: Pearson Scott Foresman.
Webb, N. M., Franke, M. L., Ing, M., Wong, J., Fernandez, C. H., Shin, N., & Turrou, A. C. (2013). Engaging with others' mathematical ideas: Interrelationships among student participation, teachers' instructional practices, and learning. International Journal of Educational Research, 63, 79-93.