Efficacy

Broadening Participation in Mathematics for English Learners with Mathematics Difficulties: A Multi-Site Impact Study

The purpose of this project is to rigorously test the efficacy of the Precision Mathematics First-Grade (PM-1) intervention on the mathematics outcomes of English learners (ELs) who face mathematics difficulties (MD). The PM-1 intervention is designed to support students with or at risk for MD in developing a robust understanding of the underlying concepts, problem-solving skills, and vocabulary of early measurement and statistical investigation.

Lead Organization(s): 
Award Number: 
2010550
Funding Period: 
Tue, 09/01/2020 to Sun, 08/31/2025
Full Description: 

Success in mathematics has major implications for student success at the public school and postsecondary levels, as well as contributing to the nation's STEM workforce and economy. While building mathematics literacy is important for all learners, it is especially critical for students most vulnerable for academic risk. Among these at-risk subgroups are English learners (ELs). By definition, ELs are not yet proficient in academic English and often come to school unprepared for the linguistic demands of disciplinary learning. Authentic and engaged content learning activities, such as early mathematics, may comprise the ideal context for acquiring the language and literacy skills ELs need. Research suggests ELs represent the fastest growing subgroup in U.S. schools. Currently, U.S. classrooms serve over five million ELs and estimated projections suggest that up to one in four students in 2025 will be an EL. Considering the rising presence of ELs, there is an urgent need to advance equity and eliminate disparities in STEM education among this group of learners who are often underserved by current practice. Overwhelming evidence suggests that ELs place well short of their non-EL peers in the area of mathematics. Therefore, the purpose of this 5-year Impact Project is to rigorously test the efficacy of the Precision Mathematics First-Grade (PM-1) intervention on the mathematics outcomes of ELs who face mathematics difficulties (MD). PM-1 is an innovative, first-grade, English-based, mathematics intervention focused on state-of-the-art technology and hands-on problem-solving activities, the PM-1 intervention is designed to support students with or at risk for MD in developing a robust understanding of the underlying concepts, problem-solving skills, and vocabulary of early measurement and statistical investigation. This study will examine student response to the PM-1 intervention based on variables such as students' initial mathematics skill levels and proficiency in English, and explore how the rate and quality of mathematics discourse opportunities for ELs may predict gains in mathematics outcomes.

This impact study will investigate the efficacy of the Precision Mathematics First-Grade (PM-1) intervention through a methodologically rigorous randomized controlled trial. The study will utilize a randomized block design, blocking on classrooms and randomly assigning first-grade English learners (ELs) who face mathematics difficulties (MD) within first-grade classrooms to one of two conditions: (a) PM-1 intervention or (b) control (business-as-usual). Approximately 900 ELs from 150 first-grade classrooms will participate. Three research aims will guide this study. Aim 1 will systematically evaluate the average effect of PM-1 on student mathematics achievement; while Aim 2 will investigate differential response to the intervention based on student-level variables, including ELs proficiency in English and pretreatment mathematics performance. In Aim 3, researchers will explore whether the frequency and quality of mathematics discourse opportunities for ELs predicts gains in mathematics achievement. Although random assignment will take place at the student level, students will be assigned to small instructional group formats for intervention delivery. Therefore, the design employs a partially nested mixed-model Time × Condition analyses to evaluate the effect of PM-1 on pretest to posttest gains in mathematics achievement (Aim 1) and differential response to PM-1 based on student characteristics (Aim 2). A random coefficients analysis that nests repeated assessments within students and PM-1 intervention groups will explore whether the rate and quality of mathematics discourse opportunities predicts ELs' gains in mathematics achievement (Aim 3).

Leveraging Simulations in Preservice Preparation to Improve Mathematics Teaching for Students with Disabilities (Collaborative Research: Jones)

This project aims to support the mathematics learning of students with disabilities through the development and use of mixed reality simulations for elementary mathematics teacher preparation. These simulations represent low-stakes opportunities for preservice teachers to practice research-based instructional strategies to support mathematics learning, and to receive feedback on their practices.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
2010298
Funding Period: 
Fri, 05/01/2020 to Tue, 04/30/2024
Full Description: 

The preparation of general education teachers to support the mathematics learning of students with disabilities is critical, as students with disabilities are overrepresented in the lower ranks of mathematics achievement. This project aims to address this need in the context of elementary mathematics teacher preparation through the development and use of mixed reality simulations. These simulations represent low-stakes opportunities for preservice teachers to practice research-based instructional strategies to support mathematics learning, and to receive feedback on their practices. Learning units that use the simulations will focus on two high leverage practices: teacher modeling of self-monitoring and reflection strategies during problem solving and using strategy instruction to teach students to support problem solving. These high-leverage teaching practices will support teachers engaging all students, including students with disabilities, in conceptually sophisticated mathematics in which students are treated as sense-makers and empowered to do mathematics in culturally meaningful ways.

The project work encompasses three primary aims. The first aim is to develop a consensus around shared definitions of high-leverage practices across the mathematics education and special education communities. To accomplish this goal, the project will convene a series of consensus-building panels with mathematics education and special education experts to develop shared definitions of the two targeted high leverage practices. This work will include engaging with current research, group discussion, and production of documents with specifications for the practices. The second aim is to develop learning units for elementary mathematics methods courses grounded in mixed reality simulation. These simulations will allow teacher candidates to enact the high leverage practices with simulated students and to receive coaching on their practice from the research team. The impact of this work will be assessed through the analysis of interviews with teacher educators implementing the units and observations and artifacts from the implementations. The third aim will be to assess the effectiveness of the simulations on teacher candidates? practices and beliefs through small-scaled randomized control trials. Teacher candidates will be randomly assigned to conditions that address the practices and make use of simulations, and a business as usual condition focused on lesson planning, student assessment, and small group discussions of the high leverage practices. The impact of the work will be assessed through the analysis of baseline and exit simulations, measures of teacher self-efficacy for teaching students with disabilities, and observations of classroom teaching in their clinical placement settings.

Developing Leaders, Transforming Practice in K-5 Mathematics: An Examination of Models for Elementary Mathematics Specialists (Collaborative Research: Lewis)

This project will study the Developing Leaders Transforming Practice (DLTP) intervention, which aims to improve teachers' instructional practices, increase student mathematics understanding and achievement.

Lead Organization(s): 
Award Number: 
1906588
Funding Period: 
Sun, 09/01/2019 to Thu, 08/31/2023
Full Description: 

Minimal rigorous research has been conducted on the effect of various supports for quality mathematics instruction and providing guidance on the development and use of Elementary Mathematics Specialists (EMSs) on student achievement. Portland Public Schools (PPS), Portland State University, and RMC Research Corporation will study the Developing Leaders Transforming Practice (DLTP) intervention, which aims to improve teachers' instructional practices, increase student mathematics understanding and achievement. The project team will evaluate the efficacy and use of EMSs by testing four implementation models that consider the various ways EMSs are integrated into schools. DLTP builds on EMS research, investigating EMSs both as elementary mathematics teachers and coaches by articulating four models and examining their efficacy for both student and teacher learning. This study has the potential to provide benefits both within and beyond PPS as it informs the preparation and use of EMSs. Determining which model is best in certain contexts provides a focus for the expansion of mathematics support.

DLTP enhances the research base by examining the effect of teacher PD on student achievement through a rigorous quasi-experimental design. The project goals will be met by addressing 4 research questions: 1) What is the effect of the intervention on teacher leadership?; 2) What is the effect of the intervention on teachers' use of research-based instructional practices?; 3) What is the effect of the intervention on a school's ability to sustain ongoing professional learning for teachers?; and 4) What is the effect of the intervention on student mathematics achievement? Twelve elementary schools in PPS will select elementary teachers to participate in the DLTP and adopt an implementation model that ranges from direct to diffuse engagement with students: elementary mathematics teacher, grade level coach, grade-level and building-level coach, or building-level coach. The research team will conduct 4 major studies that include rigorous quasi-experimental designs and a multi-method approach to address the research questions: leadership study, instructional practices study, school study, and student achievement study. Several tools will be created by the project - a leadership rubric designed to measure changes in EMS mathematics leadership because of the project and a 5-part teacher survey designed capture EMS leadership skills, pedagogical content knowledge, use of research-based practices, and school climate for mathematics learning as well as implementation issues.

Developing Leaders, Transforming Practice in K-5 Mathematics: An Examination of Models for Elementary Mathematics Specialists Collaborative Research: Davis)

This project will study the Developing Leaders Transforming Practice (DLTP) intervention, which aims to improve teachers' instructional practices, increase student mathematics understanding and achievement.

Lead Organization(s): 
Award Number: 
1906565
Funding Period: 
Sun, 09/01/2019 to Thu, 08/31/2023
Full Description: 

Minimal rigorous research has been conducted on the effect of various supports for quality mathematics instruction and providing guidance on the development and use of Elementary Mathematics Specialists (EMSs) on student achievement. Portland Public Schools (PPS), Portland State University, and RMC Research Corporation will study the Developing Leaders Transforming Practice (DLTP) intervention, which aims to improve teachers' instructional practices, increase student mathematics understanding and achievement. The project team will evaluate the efficacy and use of EMSs by testing four implementation models that consider the various ways EMSs are integrated into schools. DLTP builds on EMS research, investigating EMSs both as elementary mathematics teachers and coaches by articulating four models and examining their efficacy for both student and teacher learning. This study has the potential to provide benefits both within and beyond PPS as it informs the preparation and use of EMSs. Determining which model is best in certain contexts provides a focus for the expansion of mathematics support.

DLTP enhances the research base by examining the effect of teacher PD on student achievement through a rigorous quasi-experimental design. The project goals will be met by addressing 4 research questions: 1) What is the effect of the intervention on teacher leadership?; 2) What is the effect of the intervention on teachers' use of research-based instructional practices?; 3) What is the effect of the intervention on a school's ability to sustain ongoing professional learning for teachers?; and 4) What is the effect of the intervention on student mathematics achievement? Twelve elementary schools in PPS will select elementary teachers to participate in the DLTP and adopt an implementation model that ranges from direct to diffuse engagement with students: elementary mathematics teacher, grade level coach, grade-level and building-level coach, or building-level coach. The research team will conduct 4 major studies that include rigorous quasi-experimental designs and a multi-method approach to address the research questions: leadership study, instructional practices study, school study, and student achievement study. Several tools will be created by the project - a leadership rubric designed to measure changes in EMS mathematics leadership because of the project and a 5-part teacher survey designed capture EMS leadership skills, pedagogical content knowledge, use of research-based practices, and school climate for mathematics learning as well as implementation issues.

Developing Leaders, Transforming Practice in K-5 Mathematics: An Examination of Models for Elementary Mathematics Specialists Collaborative Research: Rigelman)

This project will study the Developing Leaders Transforming Practice (DLTP) intervention, which aims to develop teacher leaders, improve teachers' instructional practices, and increase student mathematics understanding and achievement.

Lead Organization(s): 
Award Number: 
1906682
Funding Period: 
Sun, 09/01/2019 to Thu, 08/31/2023
Project Evaluator: 
RMC Research
Full Description: 

Minimal rigorous research has been conducted on the effect of various supports for quality mathematics instruction and providing guidance on the development and use of Elementary Mathematics Specialists (EMSs) on student achievement. Portland Public Schools (PPS), Portland State University, and RMC Research Corporation will study the Developing Leaders Transforming Practice (DLTP) intervention, which aims to improve teachers' instructional practices, increase student mathematics understanding and achievement. The project team will evaluate the efficacy and use of EMSs by testing four implementation models that consider the various ways EMSs are integrated into schools. DLTP builds on EMS research, investigating EMSs both as elementary mathematics teachers and coaches by articulating four models and examining their efficacy for both student and teacher learning. This study has the potential to provide benefits both within and beyond PPS as it informs the preparation and use of EMSs. Determining which model is best in certain contexts provides a focus for the expansion of mathematics support.

DLTP enhances the research base by examining the effect of teacher PD on student achievement through a rigorous quasi-experimental design. The project goals will be met by addressing 4 research questions: 1) What is the effect of the intervention on teacher leadership?; 2) What is the effect of the intervention on teachers' use of research-based instructional practices?; 3) What is the effect of the intervention on a school's ability to sustain ongoing professional learning for teachers?; and 4) What is the effect of the intervention on student mathematics achievement? Twelve elementary schools in PPS will select elementary teachers to participate in the DLTP and adopt an implementation model that ranges from direct to diffuse engagement with students: elementary mathematics teacher, grade level coach, grade-level and building-level coach, or building-level coach. The research team will conduct 4 major studies that include rigorous quasi-experimental designs and a multi-method approach to address the research questions: leadership study, instructional practices study, school study, and student achievement study. Several tools will be created by the project - a leadership rubric designed to measure changes in EMS mathematics leadership because of the project and a 5-part teacher survey designed capture EMS leadership skills, pedagogical content knowledge, use of research-based practices, and school climate for mathematics learning as well as implementation issues.

Learning Trajectories as a Complete Early Mathematics Intervention: Achieving Efficacies of Economies at Scale

The purpose of this project is to test the efficacy of the Learning and Teaching with Learning Trajectories (LT2) program with the goal of improving mathematics teaching and thereby increasing young students' math learning. LT2 is a professional development tool and a curriculum resource intended for teachers to be used to support early math instruction and includes the mathematical learning goal, the developmental progression, and relevant instructional activities.

Lead Organization(s): 
Award Number: 
1908889
Funding Period: 
Mon, 07/01/2019 to Sun, 06/30/2024
Full Description: 

U.S. proficiency in mathematics continues to be low and early math performance is a powerful predictor of long-term academic success and employability. However, relatively few early childhood degree programs have any curriculum requirements focused on key mathematics topics. Thus, teacher professional development programs offer a viable and promising method for supporting and improving teachers' instructional approaches to mathematics and thus, improving student math outcomes. The purpose of this project is to test the efficacy of the Learning and Teaching with Learning Trajectories (LT2) program with the goal of improving mathematics teaching and thereby increasing young students' math learning. LT2 is a professional development tool and a curriculum resource intended for teachers to be used to support early math instruction. The LT2 program modules uniquely include the mathematical learning goal, the developmental progression, and relevant instructional activities. All three aspects are critical for high-quality and coherent mathematics instruction in the early grades.

This project will address the following research questions: 1) What are the medium-range effects of LT2 on student achievement and the achievement gap? 2) What are the short- and long-term effects of LT2 on teacher instructional approach, beliefs, and quality? and 3) How cost effective is the LT2 intervention relative to the original Building Blocks intervention? To address the research questions, this project will conduct a multisite cluster randomized experimental design, with 90 schools randomly assigned within school districts to either experimental or control groups. Outcome measures for the approximately 250 kindergarten classrooms across these districts will include the Research-based Elementary Math Assessment, observations of instructional quality, a questionnaire focused on teacher beliefs and practices, in addition to school level administrative data. Data will be analyzed using multi-level regression models to determine the effect of the Learning Trajectories intervention on student learning.

Testing the Efficacy of the Strategic Observation and Reflection (SOAR) for Math Professional Learning Program

The purpose of this project is to develop, implement and test a professional development program, SOAR for Math, to build capacity for mentors and teachers to improve English learner's academic language development and mathematical content understanding.

Award Number: 
1814356
Funding Period: 
Sat, 09/01/2018 to Wed, 08/31/2022
Full Description: 
Professional development is an important way for teachers who are currently in classrooms to learn about new best practices in mathematics teaching and learning and improve their practice. Little is known about what types of professional development (PD) and teacher mentoring programs support teachers' improved practices and ultimately lead to gains in student learning. The purpose of this project is to develop, implement and test a professional development program, SOAR for Math, to build capacity for mentors and teachers to improve English learner's academic language development and mathematical content understanding.
 
This study will test the efficacy of the Strategic Observation and Reflection (SOAR) for Math professional development program. The mixed methods study is designed to answer several research questions: (1) What is the impact of teachers' participation in SOAR for Math on student achievement outcomes for current and recent grade 3-6 English learner students in treatment schools? (2) What is the impact of SOAR for Math on treatment school teachers' knowledge and practices related to their academic language and literacy development instruction for current and recent English learner students, specifically scores on the Knowledge/Use Scale? (3) What is the impact of SOAR for Math on treatment mentors' knowledge and practices related to their academic language and math instruction? A randomized controlled trial will be conducted in 80 elementary schools in one California school district. Schools serving third- through sixth-grade general education students will be eligible to participate. The research team will randomly assign 40 schools to provide SOAR for Math training to mentor teachers and 40 schools to comprise a control group receiving business-as-usual professional development. Two mentors per school will participate in the study. Measures will include state math scores and a variety of observations and questionnaires to assess fidelity of implementation. Data will be analyzed using hierarchical linear modeling to account for the nested data structure.

Extending and Investigating the Impact of the High School Model-based Educational Resource (Collaborative Research: Passmore)

This project builds on a line of work that has developed and studied the Model Based Educational Resource (MBER), a year-long curriculum for high school biology. The project will generate rigorous causal evidence on how this approach to biology teaching and learning can support student learning, and foundational information on how to support high school teachers in improving their teaching. It will also provide resources to expand and update MBER to reflect the changing high school science landscape by integrating Earth Science standards into the year long sequence.

Partner Organization(s): 
Award Number: 
1814263
Funding Period: 
Wed, 08/01/2018 to Sun, 07/31/2022
Full Description: 

This project builds on a line of work that has developed and studied the Model Based Educational Resource (MBER), a year-long curriculum for high school biology. In classrooms using MBER, modeling serves as an anchoring practice that keeps the inquiry tied to the goal of making sense of the world, helping teachers to engage their students in a range of cognitive and social activities that lead to deep understanding of scientific ideas. This project will generate rigorous causal evidence on how this approach to biology teaching and learning can support student learning, and foundational information on how to support high school teachers in improving their teaching. This funding will also provide resources to expand and update MBER to reflect the changing high school science landscape by integrating Earth Science standards into the year long sequence. The study will address the general research question: What is the impact of the Model Based Educational Resource (MBER) on high school students' science achievement, and what factors influence that impact? In addition to generating important research findings, the materials revised and studied in this project will be open-source and freely available to teachers and schools.

This study addresses a significant gap in the research on next generation curriculum materials. While there is emerging agreement about the importance of instructional materials in supporting teachers in effectively engaging students in the practices of science, there is very little empirical evidence to support such claims. The goal of this project is to study the impact of the MBER program through a cluster randomized trial and expand the promise of efficacy and feasibility established in previous work. This study will be able to make causal claims by using an experimental design in which 32 high school teachers serve as their own controls, and by using multi-level modeling in the analysis. This study will advance the field's knowledge about the impact of innovative materials on student learning, measured by both project-level assessments and the state science test. Exploratory research questions will examine a) how using the MBER program develops teachers' vision of the Next Generation Science Standards, b) how student learning is mediated by the fidelity of implementation of the materials, c) how teachers interact with materials designed to be modified for their classroom context, and d) to what extent the MBER materials provide equitable opportunities to learn and close achievement gaps.

A Partnership to Adapt, Implement and Study a Professional Learning Model and Build District Capacity to Improve Science Instruction and Student Understanding (Collaborative Research: Borko)

This project will work in partnership with the Santa Clara Unified School District (SCUSD) to adapt a previously designed Professional Learning (PL) model based on the District's objectives and constraints to build the capacity of teacher leaders and a program coordinator to implement the adapted PL program. The project is examining the sustainability and scalability of a PL model that supports the development of teachers' pedagogical content knowledge and instructional practices.

Lead Organization(s): 
Award Number: 
1720930
Funding Period: 
Sun, 10/01/2017 to Thu, 09/30/2021
Full Description: 

The Lawrence Hall of Science (the Hall) and Stanford University teams have previously developed and tested the efficacy of a program of Professional Learning (PL) which is focused on improving teachers' ability to support students' ability to engage in scientific argumentation. Key components of the PL model include a week-long summer institute and follow-up sessions during the academic year that incorporate additional pedagogical input, video reflection, and planning time. In this project, the Hall and Stanford are working in partnership with the Santa Clara Unified School District (SCUSD) to adapt the PL model based on the District's objectives and constraints, to build the capacity of teacher leaders and a program coordinator to implement the adapted PL program. This will enable the District to continue to adapt and implement the program independently at the conclusion of the project. Concurrently, the project is studying the adaptability of the PL model and the effectiveness of its implementation, and is developing guidelines and tools for other districts to use in adapting and implementing the PL model in their local contexts. Thus, this project is contributing knowledge about how to build capacity in districts to lead professional learning in science that addresses the new teaching and learning standards and is responsive to the needs of their local context.

The project is examining the sustainability and scalability of a PL model that supports the development of teachers' pedagogical content knowledge and instructional practices, with a particular focus on engaging students in argument from evidence. Results from the Hall and Stanford's previous research project indicate that the PL model is effective at significantly improving teachers' and students' classroom discourse practices. These findings suggest that a version of the model, adapted to the context and needs of a different school district, has the potential to improve the teaching of science to meet the demands of the current vision of science education. Using a Design-Based Implementation Research approach, this project is (i) working with SCUSD to adapt the PL model; (ii) preparing a district project coordinator and cadre of local teacher leaders (TLs) to implement and further adapt the model; and (iii) studying the adaptation and implementation of the model. The outcomes will be: a) a scalable PL model that can be continually adapted to the objectives and constraints of a district; b) a set of activities and resources for the district to prepare and support the science teacher leaders who will implement the adapted PL program internally with other teachers; and c) knowledge about the adaptations and resources needed for the PL model to be implemented independently by other school districts. The team also is researching the impact of the program on classroom practices and student learning.


Project Videos

2020 STEM for All Video Showcase

Title: Accomplishments and Struggles in a 3-Way RPP

Presenter(s): Emily Weiss, Hilda Borko, Coralie Delhaye, Jonathan Osborne, Emily Reigh, Tricia Ringel, Craig Strang, & Krista Woodward

2019 STEM for All Video Showcase

Title: Building District Leadership in Scientific Argumentation

Presenter(s): Coralie Delhaye, Emily Reigh, & Emily Weiss

2018 STEM for All Video Showcase


Integration of Engineering Design and Life Science: Investigating the Influence of an Intervention on Student Interest and Motivation in STEM Fields

This project will investigate the integration of engineering design, practices, and thinking into middle school life science curriculum while providing opportunities for students to foster knowledge of and increase interest in life and biosciences. The project will specifically respond to the need to create, implement, and evaluate a model intervention that will advance the knowledge base for establishing and retaining underrepresented minorities in STEM fields.

Lead Organization(s): 
Award Number: 
1721141
Funding Period: 
Fri, 09/01/2017 to Tue, 08/31/2021
Full Description: 

This project will investigate the integration of engineering design, practices, and thinking into middle school life science curriculum while providing opportunities for students to foster knowledge of and increase interest in life and biosciences. The project will specifically respond to the need to create, implement, and evaluate a model intervention that will advance the knowledge base for establishing and retaining underrepresented minorities in STEM fields. Specifically, the project will partner with middle school science teachers from two local school corporations, STEM university faculty members and undergraduate engineering students, and university-based outreach coordinators from a minorities engineering program, the office of future engineers, and women in engineering program. Through this combined effort, both school corporations that serve underserved, culturally diverse, and socioeconomically disadvantaged students in rural communities; will have broad-based support for engaging 36 teachers and 3000 students in integrated life science with engineering design.

The project will employ a mixed methods research design incorporating both qualitative and quantitative approaches for data collection and analyses. The research team will conduct quantitative analyses by using Hierarchical Linear Modeling to determine the extent to which integrating life science with engineering design and thinking impact student learning of life science concepts and interest in life and biosciences. Qualitative approaches, including discourse analysis, will be used to delve deeper into student learning of the targeted life science concepts. Through this research, the project will advance evidence-based understanding of learning, enhance the theoretical models of student life science learning, and merge and extend the successes of previous studies by using the faculty expertise in effective approaches in engineering integration in K-12 science classrooms. Specifically, concept assessments, interest surveys, recordings of classroom discourse, student artifacts (e.g., design reports), interviews, and classroom observations will be used as data sources. Outcomes from the project will advance the knowledge base for establishing and retaining underrepresented minorities in STEM fields. The life STEM focused design tasks will be disseminated through an online peer-reviewed digital library available for use across the U.S. and beyond. Along with the design-based tasks on this website; results from the intervention model will be disseminated through electronic and print media to inform researchers, educators, administrators, and policy makers who play critical roles in enhancing student learning of and interest in STEM, about pathways to broadening participation in STEM.

Pages

Subscribe to Efficacy