Qualitative

Testing the Efficacy of the Strategic Observation and Reflection (SOAR) for Math Professional Learning Program

The purpose of this project is to develop, implement and test a professional development program, SOAR for Math, to build capacity for mentors and teachers to improve English learner's academic language development and mathematical content understanding.

Award Number: 
1814356
Funding Period: 
Sat, 09/01/2018 to Wed, 08/31/2022
Full Description: 
Professional development is an important way for teachers who are currently in classrooms to learn about new best practices in mathematics teaching and learning and improve their practice. Little is known about what types of professional development (PD) and teacher mentoring programs support teachers' improved practices and ultimately lead to gains in student learning. The purpose of this project is to develop, implement and test a professional development program, SOAR for Math, to build capacity for mentors and teachers to improve English learner's academic language development and mathematical content understanding.
 
This study will test the efficacy of the Strategic Observation and Reflection (SOAR) for Math professional development program. The mixed methods study is designed to answer several research questions: (1) What is the impact of teachers' participation in SOAR for Math on student achievement outcomes for current and recent grade 3-6 English learner students in treatment schools? (2) What is the impact of SOAR for Math on treatment school teachers' knowledge and practices related to their academic language and literacy development instruction for current and recent English learner students, specifically scores on the Knowledge/Use Scale? (3) What is the impact of SOAR for Math on treatment mentors' knowledge and practices related to their academic language and math instruction? A randomized controlled trial will be conducted in 80 elementary schools in one California school district. Schools serving third- through sixth-grade general education students will be eligible to participate. The research team will randomly assign 40 schools to provide SOAR for Math training to mentor teachers and 40 schools to comprise a control group receiving business-as-usual professional development. Two mentors per school will participate in the study. Measures will include state math scores and a variety of observations and questionnaires to assess fidelity of implementation. Data will be analyzed using hierarchical linear modeling to account for the nested data structure.

Developing Preservice Teachers' Capacity to Teach Students with Learning Disabilities in Algebra I

Project researchers are training pre-service teachers to tutor students with learning disabilities in Algebra 1, combining principles from special education, mathematics education, and cognitive psychology. The trainings emphasize the use of gestures and strategic questioning to support students with learning disabilities and to build students’ understanding in Algebra 1.

Project Email: 
Lead Organization(s): 
Award Number: 
1813903
Funding Period: 
Wed, 08/01/2018 to Sat, 07/31/2021
Full Description: 

This project is implementing a program to train pre-service teachers to tutor students with learning disabilities in Algebra 1, combining principles from special education, mathematics education, and cognitive psychology. The project trains tutors to utilize gestures and strategic questioning to support students with LD to build connections between procedural knowledge and conceptual understanding in Algebra 1, while supporting students’ dispositions towards doing mathematics. The training will prepare tutors to address the challenges that students with LD often face—especially challenges of working memory and processing—and to build on their strengths as they engage with Algebra 1. The project will measure changes in tutors’ ability to use gestures and questioning to support the learning of students with LD during and after the completion of our training. It will also collect and analyze data on the knowledge and dispositions of students with LD in Algebra 1 for use in the ongoing refinement of the training and in documenting the impact of the training program.

Science Communities of Practice Partnership

This project will study implementation of an effective professional learning model for elementary science teachers that includes teacher leaders, administrators and university educators in a system perspective for improving science instruction in ways that make it sustainable.

Award Number: 
1813012
Funding Period: 
Wed, 08/01/2018 to Sun, 07/31/2022
Full Description: 

This project will study implementation of an effective professional learning model for elementary science teachers that includes teacher leaders, administrators and university educators in a system perspective for improving science instruction in ways that make it sustainable. The working model involves reciprocal communities of practice, which are groups of teachers, leaders and administrators that focus on practical tasks and how to achieve them across these stakeholder perspectives. The project will provide evidence about the specific components of the professional development model that support sustainable improvement in science teaching, will test the ways that teacher ownership and organizational conditions mediate instructional change, and will develop four tools for facilitating the teacher learning and the accompanying capacity building. In this way, the project will produce practical knowledge and tools necessary for other school districts nationwide to create professional learning that is tailored to their contexts and therefore sustainable.

This study posits that communication among district teachers, teacher leaders, and administrators, and a sense of ownership for improved instruction among teachers can support sustainable change. As such, it tests a model that fosters communication and ownership through three reciprocal communities of practice--one about district leadership including one teacher per school, coaches and university faculty; another about lesson study including teachers, coaches and faculty; and a third about instructional innovation including teachers and administrators, facilitated by coaches. The research design seeks to inform what the communities of practice add to the effects in a quasi-experimental study involving 72 third to fifth grade teachers and 6500 students in four urban school districts. Mixed methodologies will be used to examine shifts in science teaching over three years, testing the professional development model and the mediating roles of reform ownership and organizational conditions.

Strengthening Data Literacy across the Curriculum (SDLC)

This project is developing and studying high school curriculum modules that integrate social justice topics with statistical data investigations to promote skills and interest in data science among underrepresented groups in STEM.

Award Number: 
1813956
Funding Period: 
Sun, 07/01/2018 to Wed, 06/30/2021
Full Description: 

The Strengthening Data Literacy across the Curriculum (SDLC) project is an exploratory/early stage design and development effort that aims to promote understanding of core statistical concepts and interest in quantitative data analysis among high school students from underrepresented groups in STEM. Led by a collaboration of researchers and developers at Education Development Center (EDC), statistics educators at California Polytechnic State University (Cal Poly), and technology developers at The Concord Consortium, the project is creating and studying a set of curriculum modules targeted to high school students who are taking mathematics or statistics classes that are not at advanced-placement (AP) levels. Iteratively developed and tested in collaboration with high school statistics and social studies teachers, the modules consist of applied data investigations structured around a four-step data investigation cycle that engage students in explorations of authentic social science issues using large-scale data sets from the U.S. Census Bureau. The project hypothesizes that students who engage in guided investigations using data visualization tools to explore and visualize statistical concepts may develop deeper understandings of these concepts as well as the data investigation process. Similarly, high school students – particularly those from historically marginalized groups who are underrepresented in STEM fields – may develop greater interest in statistics when they can use data to examine patterns of social and economic inequality and questions related to social justice.

One module, Investigating Income Inequality in the U.S., focuses on describing, comparing, and making sense of quantitative variables. Students deepen their understanding of this content by investigating questions such as: How have incomes for higher- and lower-income individuals in the U.S. changed over time? How much income inequality exists between males and females in the U.S.? Does education explain the wage gap between males and females? Another module, Investigating Immigration to the U.S., focuses on describing, comparing, and making sense of categorical variables. Students investigate questions such as: Are there more immigrants in the U.S. today than in previous years? Where have immigrants to the U.S. come from, now and in the past? Are immigrants as likely as the U.S. born to be participating in the labor force, after adjusting for education? Students conduct these analyses using the Common Online Data Analysis Platform (CODAP), an open-source set of tools that supports data visualization and conceptual understanding of statistical ideas over calculations. Lessons encourage collaborative inquiry and provide students with experiences in multivariable analysis—an important domain that is underemphasized in current high school mathematics and statistics curricula but critical for analyzing data in a big-data world.

The project is using a mixed methods approach to study three primary research questions: 1) What is the feasibility of implementing SDLC modules, and what supports may teachers and students need to use the modules? 2) In what ways may different features and components of the SDLC modules help to promote positive student learning and interest outcomes? 3) To what extent do students show greater interest in statistics and data analysis, as well as improved understandings of target statistical concepts, after module use? To investigate these questions, the project has worked with 12 mathematics and six social studies teachers in diverse public high schools in Massachusetts and California to conduct iterative research with over 600 students. Through this work, the project aims to build knowledge of curriculum-based approaches that prepare and attract more diverse populations to data science fields.

Improving Multi-Dimensional Assessment and Instruction: Building and Sustaining Elementary Science Teachers' Capacity through Learning Communities (Collaborative Research: Lehman)

The main goal of this project is to better understand how to build and sustain the capacity of elementary science teachers in grades 3-5 to instruct and formatively assess students in ways that are aligned with contemporary science education frameworks and standards. To achieve this goal, the project will use classroom-based science assessment as a focus around which to build teacher capacity in science instruction and three-dimensional learning in science.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1813938
Funding Period: 
Sun, 07/01/2018 to Thu, 06/30/2022
Full Description: 

This is an Early-Stage Design and Development collaborative effort submitted to the assessment strand of the Discovery Research PreK-12 (DRK-12) Program. Its main goal is to better understand how to build and sustain the capacity of elementary science teachers in grades 3-5 to instruct and formatively assess students in ways that are aligned with contemporary science education frameworks and standards. To achieve this goal, the project will use classroom-based science assessment as a focus around which to build teacher capacity in science instruction and three-dimensional learning in science. The three dimensions will include disciplinary core ideas, science and engineering practices, and crosscutting concepts. These dimensions are described in the Framework for K-12 Science Education (National Research Council; NRC, 2012), and the Next Generation Science Standards (NGSS; NGSS Lead States, 2013). The project will work closely with teachers to co-develop usable assessments and rubrics and help them to learn about three-dimensional assessment and instruction. Also, the project will work with teachers to test the developed assessments in diverse settings, and to create an active, online community of practice.

The two research questions will be: (1) How well do these assessments function with respect to aspects of validity for classroom use, particularly in terms of indicators of student proficiency, and tools to support teacher instructional practice?; and (2) In what ways do providing these assessment tasks and rubrics, and supporting teachers in their use, advance teachers' formative assessment practices to support multi-dimensional science instruction? The research and development components of this project will produce assessments and rubrics, which can directly impact students and teachers in the districts and states that have adopted the NGSS, as well as those that have embraced the vision of science teaching and learning embodied in the NRC Framework. The project will consist of five major tasks. First, the effort will iteratively develop assessments and rubrics for formative use, using an evidence-centered design approach. Second, it will collect data from evidence-based revision and redesign of the assessments from teachers piloting the assessments and rubrics, project cognitive laboratory studies with students, and an external review of the assessments design products. Third, it will study teachers' classroom use of assessments to understand and document how they blend assessment and instruction. The project will use pre/post questionnaires, video recordings, observation field notes, and pre/post interviews. Fourth, the study will build the capacity of participating teachers. Teacher Collaborators (n=9) will engage in participatory design of the assessment tasks and act as technical assistants to the overall implementation process. Teacher Implementers (n=15) will use the assessments formatively as part of their instructional practice. Finally, the work will develop a community of learners through the development of a technical assistance infrastructure, and leveraging teacher expertise to formatively assess students' work, using the assessments designed to be diagnostic and instructionally informative. External reviewers and an advisory board will provide formative feedback on the project's processes and summative evaluation of the project's results. The main outcomes of this endeavor will be prototypes of elementary science multi-dimensional assessments and new knowledge for the field on the underlying theory for developing teachers' capacity for engaging in multi-dimensional science instruction, learning, and assessment.

Professional Development for K-12 Science Teachers in Linguistically Diverse Classrooms

This project will engage science teachers in a sustained professional development (PD) program embedded in an afterschool science program designed for a linguistically diverse group of English learners (ELs).

Award Number: 
2001688
Funding Period: 
Tue, 05/01/2018 to Sat, 04/30/2022
Full Description: 

This project will engage science teachers in a sustained professional development (PD) program embedded in an afterschool science program designed for a linguistically diverse group of English learners (ELs). The project targets science teachers (chemistry, physics, biology, and earth science) who teach in a high school that includes refugees from Myanmar, Central America, and Africa. Roughly 20% of the students are classified as ELs, representing almost 20 different linguistic groups, including a variety of Asian, Spanish, and Arabic languages. The fundamental issue that the project seeks to address is the design of science learning environments to facilitate ELs' learning in linguistically diverse high school classrooms. Research on science education for ELs has recommended several effective teaching approaches, such as building on students' diverse and rich resources, engaging students in authentic science learning practices, and encouraging and valuing flexible use of multiple languages. However, previously most research has focused on teaching speakers of Spanish in elementary and middle school level science classrooms in which a majority of ELs speak the same language. Furthermore, while many PD programs supporting science education for ELs provide a short-term workshop and/or newly designed curriculum and curriculum guide, there is a lack of PD models that engage teachers in a sustained community of practice through collaboration between researchers and teachers.

The project's primary goal includes broadening participation with direct impact on 14 science teachers, who will impact over 2000 students, including over 450 ELs, during the project implementation period. The project provides a sustained model of the PD program which further impacts EL students of teachers who participated in the various phases of the project. The project has a potential to make an impact on ELs and high school science teachers of ELs in three different ways. First, by generating PD materials that include effective teaching materials and instructional practices for ELs, which can be used by other educators situated in similar educational contexts. Second, by giving presentations and publish papers that communicate findings of the project to academic communities. These outputs can impact other researchers who would like to design PD programs to foster ELs' science learning. Third, by implementing the developed and tested PD program in a larger scale. The implementation of the project will build capacity to conduct a larger PD project to impact more teachers and students. These anticipated outputs and outcomes will provide valuable resources for researcher and practitioners looking to support ELs' science learning and steps forward to equity. Finally, the project team and two cohorts of science teachers will co-design a school-wide science teacher PD to transform science teaching materials and practices of non-participating teachers.

This project was previously funded under award #1813937.

CAREER: Supporting Elementary Science Teaching and Learning by Integrating Uncertainty Into Classroom Science Investigations

The goal of this study is to improve elementary science teaching and learning by developing, testing, and refining a framework and set of tools for strategically incorporating forms of uncertainty central to scientists' sense-making into students' empirical learning.

Lead Organization(s): 
Award Number: 
1749324
Funding Period: 
Fri, 06/01/2018 to Wed, 05/31/2023
Full Description: 

The goal of this study will be to improve elementary science teaching and learning by developing, testing, and refining a framework and set of tools for strategically incorporating forms of uncertainty central to scientists' sense-making into students' empirical learning. The framework will rest on the notion that productive uncertainty should be carefully built into students' empirical learning experiences in order to support their engagement in scientific practices and understanding of disciplinary ideas. To re-conceptualize the role of empirical investigations, the study will focus on the transitions between the experiences and processes students seek to understand, classroom investigations, evidence, and explanatory models as opportunities for sense-making, and how uncertainty can be built into these transitions. The project's underlying assumption is that carefully implementing these forms of uncertainty will help curriculum developers and teachers avoid the oversimplified investigations that are prevalent in K-8 classrooms that stand in stark contrast to authentic science learning and the recommendations of the Framework for K-12 Science Education (National Research Council, 2012). Accordingly, the project will seek to develop curriculum design guidelines, teacher tools, professional development supports, and four elaborated investigations, including sets of lessons, videos, and assessments that embed productive uncertainty for second and fifth graders and designed for use with linguistically, culturally, and socio-economically diverse students.

The hypothesis of this work is that if specific forms of scientific uncertainty are carefully selected, and if teachers can implement these forms of uncertainty, elementary students will have more robust opportunities to develop disciplinary practices and ideas in ways consistent with the Next Generation Science Standards (NGSS) (Lead States, 2013). Employing Design-Based Research, the three research questions will be: (1) What opportunities for sense-making do elementary school empirical investigations afford where we might strategically build uncertainty?; (2) How can we design learning environments where uncertainty in empirical investigations supports opportunities for learning?; and (3) In classrooms with sustained opportunities to engage with uncertainty in empirical investigations, what progress do students make in content understandings and the practices of argumentation, explanation, and investigation? The work will consist of three design cycles: Design Cycle 1 will involve two small groups of six teachers in adapting their curricula to incorporate uncertainty, then describe how students engage around uncertainty in empirical investigations. Design Cycle 2 will involve the same small groups in implementing and refining task structures, tools, and teacher instructional strategies. In Design Cycle 3, teachers and researchers will further refine lesson materials, assessments, and supports. The project will partner with one school district and engage in design research with groups of teachers to develop: (1) a research-based description, with exemplars of opportunities for student sense-making within empirical investigations at both early and upper elementary grades; (2) a set of design principles and tools that allow teachers to elicit and capitalize on sense-making about uncertainty in investigations; and (3) four elementary investigations elaborated to incorporate and exemplify the first two products above. These materials will be disseminated through a website, and established networks for supporting implementation of the NGSS. An advisory board will oversee project progress and conduct both formative and summative evaluation.

Measuring Early Mathematical Reasoning Skills: Developing Tests of Numeric Relational Reasoning and Spatial Reasoning

The primary aim of this study is to develop mathematics screening assessment tools for Grades K-2 over the course of four years that measure students' abilities in numeric relational reasoning and spatial reasoning. The team of researchers will develop Measures of Mathematical Reasoning Skills system, which will contain Tests of Numeric Relational Reasoning (T-NRR) and Tests of Spatial Reasoning (T-SR).

Award Number: 
1721100
Funding Period: 
Fri, 09/15/2017 to Tue, 08/31/2021
Full Description: 

Numeric relational reasoning and spatial reasoning are critical to success in later mathematics coursework, including Algebra 1, a gatekeeper to success at the post-secondary level, and success in additional STEM domains, such as chemistry, geology, biology, and engineering. Given the importance of these skills for later success, it is imperative that there are high-quality screening tools available to identify students at-risk for difficulty in these areas. The primary aim of this study is to develop mathematics screening assessment tools for Grades K-2 over the course of four years that measure students' abilities in numeric relational reasoning and spatial reasoning. The team of researchers will develop Measures of Mathematical Reasoning Skills system, which will contain Tests of Numeric Relational Reasoning (T-NRR) and Tests of Spatial Reasoning (T-SR). The measures will be intended for use by teachers and school systems to screen students to determine who is at-risk for difficulty in early mathematics, including students with disabilities. The measures will help provide important information about the intensity of support that may be needed for a given student. Three forms per grade level will be developed for both the T-NRR and T-SR with accompanying validity and reliability evidence collected. The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

The development of the T-NRR and T-SR measures will follow an iterative process across five phases. The phases include (1) refining the construct; (2) developing test specifications and item models; (3) developing items; (4) field testing the items; and (5) conducting validity studies. The evidence collected and evaluated during each phase will contribute to the overall evaluation of the reliability of the measures and the validity of the interpretations made using the measures. Item models, test specifications, and item development will be continuously evaluated and refined based on data from cognitive interviews, field tests, and reviews by mathematics educators, teachers of struggling students, teachers of culturally and linguistically diverse populations, and a Technical Advisory Board. In the final phase of development of the T-NRR and T-SR, reliability of the results will be estimated and multiple sources of validity evidence will be collected to examine the concurrent and predictive relation with other criterion measures, classification accuracy, and sensitivity to growth. Approximately 4,500 students in Grades K-2 will be involved in all phases of the research including field tests and cognitive interviews. Data will be analyzed using a two-parameter IRT model to ensure item and test form comparability.


Project Videos

2020 STEM for All Video Showcase

Title: Measuring Early Mathematical Reasoning Skills

Presenter(s): Leanne Ketterlin Geller


Developing Teacher Noticing in Engineering in an Online Professional Development Program

This project will research how elementary (K-5) teachers in the Teacher Engineering Education Program (TEEP) program progress in one particular aspect of responsive teaching, noticing student thinking. Project research will also contribute to literature on how to support responsive teaching in web-based environments, expanding understanding of how design principles and features developed in in-person professional development settings can be implemented online.

Lead Organization(s): 
Award Number: 
1720334
Funding Period: 
Fri, 09/01/2017 to Mon, 08/31/2020
Full Description: 

The project will research how elementary (K-5) teachers in the Teacher Engineering Education Program (TEEP) program progress in one particular aspect of responsive teaching, noticing student thinking. TEEP includes four graduate-level courses that help them learn engineering content and pedagogical approaches. There has been little investigation of teacher professional development in engineering design. The work that has been done focuses on increasing teachers' content knowledge and familiarity in engineering. Most teacher professional development and research focus on teachers learning engineering content and process, with less attention on helping teachers develop new instructional practices necessary to help students navigate the complex, ill-defined problems in engineering. TEEP focuses on helping teachers develop practices of responsive teaching in engineering design, where teachers base their instructional moves on what they notice in their students are doing and saying. Project research will also contribute to literature on how to support responsive teaching in web-based environments, expanding understanding of how design principles and features developed in in-person professional development settings can be implemented online. The project will refine a program for engineering teachers nationwide, identify key features that are effective in developing teachers' practice, and create video resources for other professional development programs to use.

The project will address three research questions: (1) What do beginning engineering teachers notice in students' engineering design work? (2) What shifts occur in teachers' noticing over the course of a professional development program focused on responsive teaching and how do these shifts correlate with key features of the program? (3) What shifts occur in how teachers' talk about their goals for students' engineering and their instructional practice? The project will conduct independent analyses from two cohorts of teachers of three data streams: pre-post interviews about practice; teacher-captured classroom videos; video-stimulated interviews, and teachers' coursework. The analyses will then connect these analyses to address the research questions. Videocases of students' engineering will be disseminated for other teacher educators to use in supporting teacher noticing. The research outcomes of the research will not only advance our understandings of teacher learning, but will provide evidence that teachers can recognize, value, and leverage students' diverse resources for engineering. Research on the TEEP program will also provide much-needed empirical support on whether and how online programs can be effective for teachers' instructional practice.

A Partnership to Adapt, Implement and Study a Professional Learning Model and Build District Capacity to Improve Science Instruction and Student Understanding (Collaborative Research: Borko)

This project will work in partnership with the Santa Clara Unified School District (SCUSD) to adapt a previously designed Professional Learning (PL) model based on the District's objectives and constraints to build the capacity of teacher leaders and a program coordinator to implement the adapted PL program. The project is examining the sustainability and scalability of a PL model that supports the development of teachers' pedagogical content knowledge and instructional practices.

Lead Organization(s): 
Award Number: 
1720930
Funding Period: 
Sun, 10/01/2017 to Thu, 09/30/2021
Full Description: 

The Lawrence Hall of Science (the Hall) and Stanford University teams have previously developed and tested the efficacy of a program of Professional Learning (PL) which is focused on improving teachers' ability to support students' ability to engage in scientific argumentation. Key components of the PL model include a week-long summer institute and follow-up sessions during the academic year that incorporate additional pedagogical input, video reflection, and planning time. In this project, the Hall and Stanford are working in partnership with the Santa Clara Unified School District (SCUSD) to adapt the PL model based on the District's objectives and constraints, to build the capacity of teacher leaders and a program coordinator to implement the adapted PL program. This will enable the District to continue to adapt and implement the program independently at the conclusion of the project. Concurrently, the project is studying the adaptability of the PL model and the effectiveness of its implementation, and is developing guidelines and tools for other districts to use in adapting and implementing the PL model in their local contexts. Thus, this project is contributing knowledge about how to build capacity in districts to lead professional learning in science that addresses the new teaching and learning standards and is responsive to the needs of their local context.

The project is examining the sustainability and scalability of a PL model that supports the development of teachers' pedagogical content knowledge and instructional practices, with a particular focus on engaging students in argument from evidence. Results from the Hall and Stanford's previous research project indicate that the PL model is effective at significantly improving teachers' and students' classroom discourse practices. These findings suggest that a version of the model, adapted to the context and needs of a different school district, has the potential to improve the teaching of science to meet the demands of the current vision of science education. Using a Design-Based Implementation Research approach, this project is (i) working with SCUSD to adapt the PL model; (ii) preparing a district project coordinator and cadre of local teacher leaders (TLs) to implement and further adapt the model; and (iii) studying the adaptation and implementation of the model. The outcomes will be: a) a scalable PL model that can be continually adapted to the objectives and constraints of a district; b) a set of activities and resources for the district to prepare and support the science teacher leaders who will implement the adapted PL program internally with other teachers; and c) knowledge about the adaptations and resources needed for the PL model to be implemented independently by other school districts. The team also is researching the impact of the program on classroom practices and student learning.


Project Videos

2020 STEM for All Video Showcase

Title: Accomplishments and Struggles in a 3-Way RPP

Presenter(s): Emily Weiss, Hilda Borko, Coralie Delhaye, Jonathan Osborne, Emily Reigh, Tricia Ringel, Craig Strang, & Krista Woodward

2019 STEM for All Video Showcase

Title: Building District Leadership in Scientific Argumentation

Presenter(s): Coralie Delhaye, Emily Reigh, & Emily Weiss

2018 STEM for All Video Showcase


Pages

Subscribe to Qualitative