Measuring the Effectiveness of Middle School STEM Innovation and Engineering Design Curricula

Inclusion of engineering in the Next Generation Science Standards has led to increased opportunities for K-12 students to learn engineering related concepts and skills, and learn about engineering career paths. However, a persistent challenge is the lack of high-quality, research-based engineering curricular resources that align with science and math education standards. Further, the opportunities for K-12 students to also learn about manufacturing and how manufacturing is related to engineering, math, and science are limited. Researchers from Georgia Tech have developed a three-year middle school Engineering and Technology course sequence that introduces students to advanced manufacturing tools such as computer aided design (CAD) and 3D printing, incorporates engineering concepts such as pneumatics, robotics and aeronautics, increases student awareness of career paths, and addresses the concerns of technical employers wanting workers with problem solving, teamwork, and communication skills. This DRK-12 impact study project will investigate the effectiveness of STEM-Innovation and Design (STEM-ID) curricula in approximately 29 middle schools, targeting 29 engineering teachers and approximately 5,000 students across middle grades in Georgia. This impact research study will determine whether STEM-ID courses are equally effective across different demographic groups and school environments under normal implementation conditions and whether the courses have the potential to positively impact a vast number of students around the country, particularly students who have struggled to stay engaged with their STEM education. It is a critical part of a larger effort to move the STEM-ID curricula, developed with NSF support, from the research lab to large-scale practice in schools.

To facilitate large-scale implementation, the project will transfer all curriculum and teacher support materials to an online dissemination site, develop just-in-time teacher support materials to embed within the curriculum, create an online professional development platform, and conduct professional learning in multiple areas of the state. The project team will then assess the transferability of the STEM-ID curricula and identify teacher outcomes that affect the implementation. They will also examine the generalizability of the curriculum by measuring student outcomes in STEM academic achievement and on social-emotional scales. The project’s research questions consider 1) contextual factors that influence scaling; 2) the fidelity of implementation, curriculum adaptations and sustainability; 3) the effects of professional development on teachers’ engineering self-efficacy and instructional practices; 4) the effect of participation on student academic performance in mathematics and science; 5) the effect of participation on student social-emotional outcomes; and 5) the relationship between the way STEM-ID is implemented and the student outcomes.  To examine the effects of STEM-ID on achievement and achievement growth, the investigators will use a multilevel growth model and mediation analysis to explore if the intervention’s effect on achievement was mediated by students’ engagement, academic self-efficacy, and/or interest in STEM. Additionally, drawing upon Century and Cassata’s Fidelity of Implementation framework (FOI), they will examine the array of factors that influence implementation of the STEM-ID curricula across diverse school settings.

Project Videos

2022 STEM for All Video Showcase

Title: Impact Study: STEM-Innovation and Design (STEM-ID) Courses Presenter(s): Meltem Alemdar, Jasmine Choi, Tim Cone, Jessica Gale, Roxanne Moore, Abeera Rehmat, Jeff Rosen, Marion Usselman, & Julia Varnedoe