Mixed Methods

A Partnership to Adapt, Implement and Study a Professional Learning Model and Build District Capacity to Improve Science Instruction and Student Understanding (Collaborative Research: Borko)

This project will work in partnership with the Santa Clara Unified School District (SCUSD) to adapt a previously designed Professional Learning (PL) model based on the District's objectives and constraints to build the capacity of teacher leaders and a program coordinator to implement the adapted PL program. The project is examining the sustainability and scalability of a PL model that supports the development of teachers' pedagogical content knowledge and instructional practices.

Lead Organization(s): 
Award Number: 
1720930
Funding Period: 
Sun, 10/01/2017 to Thu, 09/30/2021
Full Description: 

The Lawrence Hall of Science (the Hall) and Stanford University teams have previously developed and tested the efficacy of a program of Professional Learning (PL) which is focused on improving teachers' ability to support students' ability to engage in scientific argumentation. Key components of the PL model include a week-long summer institute and follow-up sessions during the academic year that incorporate additional pedagogical input, video reflection, and planning time. In this project, the Hall and Stanford are working in partnership with the Santa Clara Unified School District (SCUSD) to adapt the PL model based on the District's objectives and constraints, to build the capacity of teacher leaders and a program coordinator to implement the adapted PL program. This will enable the District to continue to adapt and implement the program independently at the conclusion of the project. Concurrently, the project is studying the adaptability of the PL model and the effectiveness of its implementation, and is developing guidelines and tools for other districts to use in adapting and implementing the PL model in their local contexts. Thus, this project is contributing knowledge about how to build capacity in districts to lead professional learning in science that addresses the new teaching and learning standards and is responsive to the needs of their local context.

The project is examining the sustainability and scalability of a PL model that supports the development of teachers' pedagogical content knowledge and instructional practices, with a particular focus on engaging students in argument from evidence. Results from the Hall and Stanford's previous research project indicate that the PL model is effective at significantly improving teachers' and students' classroom discourse practices. These findings suggest that a version of the model, adapted to the context and needs of a different school district, has the potential to improve the teaching of science to meet the demands of the current vision of science education. Using a Design-Based Implementation Research approach, this project is (i) working with SCUSD to adapt the PL model; (ii) preparing a district project coordinator and cadre of local teacher leaders (TLs) to implement and further adapt the model; and (iii) studying the adaptation and implementation of the model. The outcomes will be: a) a scalable PL model that can be continually adapted to the objectives and constraints of a district; b) a set of activities and resources for the district to prepare and support the science teacher leaders who will implement the adapted PL program internally with other teachers; and c) knowledge about the adaptations and resources needed for the PL model to be implemented independently by other school districts. The team also is researching the impact of the program on classroom practices and student learning.


Project Videos

2020 STEM for All Video Showcase

Title: Accomplishments and Struggles in a 3-Way RPP

Presenter(s): Emily Weiss, Hilda Borko, Coralie Delhaye, Jonathan Osborne, Emily Reigh, Tricia Ringel, Craig Strang, & Krista Woodward

2019 STEM for All Video Showcase

Title: Building District Leadership in Scientific Argumentation

Presenter(s): Coralie Delhaye, Emily Reigh, & Emily Weiss

2018 STEM for All Video Showcase


Project MAPLE: Makerspaces Promoting Learning and Engagement

The project plans to develop and study a series of metacognitive strategies that support learning and engagement for struggling middle school students during makerspace experiences. The study will focus narrowly on establishing a foundational understanding of how to ameliorate barriers to engaging in design learning through the use of metacognitive strategies.

Award Number: 
1721236
Funding Period: 
Fri, 09/01/2017 to Sat, 08/31/2019
Full Description: 

The project plans to develop and study a series of metacognitive strategies that support learning and engagement for struggling middle school students during makerspace experiences. The makerspace movement has gained recognition and momentum, which has resulted in many schools integrating makerspace technologies and related curricular practices into the classroom. The study will focus narrowly on establishing a foundational understanding of how to ameliorate barriers to engaging in design learning through the use of metacognitive strategies. The project plans to translate and apply research on the use of metacognitive strategies in supporting struggling learners to develop approaches that teachers can implement to increase opportunities for students who are the most difficult to reach academically. Project strategies, curricula, and other resources will be disseminated through existing outreach websites, research briefs, peer-reviewed publications for researchers and practitioners, and a webinar for those interested in middle-school makerspaces for diverse learners.

The research will address the paucity of studies to inform practitioners about what pedagogical supports help struggling learners engage in these makerspace experiences. The project will focus on two populations of struggling learners in middle schools, students with learning disabilities, and students at risk for academic failure. The rationale for focusing on metacognition within makerspace activities comes from the literature on students with learning disabilities and other struggling learners that suggests that they have difficulty with metacognitive thinking. Multiple instruments will be used to measure metacognitive processes found to be pertinent within the research process. The project will tentatively focus on persistence (attitudes about making), iteration (productive struggle) and intentionality (plan with incremental steps). The work will result in an evidence base around new instructional practices for middle school students who are struggling learners so that they can experience more success during maker learning experiences.

Integrating Chemistry and Earth Science

This project will design, develop, and test a new curriculum unit for high school chemistry courses that is organized around the question, "How does chemistry shape where I live?" The new unit will integrate relevant Earth science data, scientific practices, and key urban environmental research findings with the chemistry curriculum to gain insights into factors that support the approach to teaching and learning advocated by current science curriculum standards.

Award Number: 
1721163
Funding Period: 
Tue, 08/15/2017 to Wed, 07/31/2019
Full Description: 

This Integrating Chemistry and Earth science (ICE) project will design, develop, and test a new curriculum unit for high school chemistry courses that is organized around the question, "How does chemistry shape where I live?" The new unit will integrate relevant Earth science data, scientific practices, and key urban environmental research findings with the chemistry curriculum to gain insights into factors that support the approach to teaching and learning advocated by current science curriculum standards. The overarching goal of the project is to develop teacher capacity to teach and evaluate student abilities to use the practices of scientists and concepts from Earth science and chemistry to understand important phenomena in their immediate, familiar environments. The project has the potential to serve as a model for how to make cutting edge science directly accessible to all students. The project is a collaborative effort that engages scientists, science education researchers, curriculum developers, school curriculum and instruction leaders, and science teachers in the longer term challenge of infusing Earth science concepts and practices across the core high school science courses.

Current guidelines and standards for science education promote learning that engages students in three interrelated dimensions: disciplinary core ideas, scientific practices, and crosscutting ideas. This project is guided by the hypothesis that when provided sustained opportunities to engage in three-dimensional learning experiences, in an integrated Earth science and chemistry context, students will improve in their ability to demonstrate the coordination of disciplinary core ideas, scientific practices, and crosscutting concepts when solving problems and developing explanations related to scientific phenomena. This project will employ a design based research approach, and during the two development-enactment-analysis-and-redesign cycles, the project team will collect student assessment data, teacher interview data, observational data from lessons, teacher surveys, and reflective teacher logbooks. These collected data will provide information about how teachers implement the lessons, what students do during the lessons, and what students learn from them that will lead to better design and a better understanding of student learning. This information will be used to inform the modification of lessons from cycle to cycle, and to inform the professional development materials for teachers. The research agenda for the project is guided by the following questions: 1. What are the design features of ICE lessons that support teachers in enacting three-dimensional instruction within the context of their classroom? 2. What are the design features of embedded three-dimensional assessments that yield useful classroom data for teachers and researchers regarding their students' abilities to integrate core ideas, scientific practices, and crosscutting concepts? 3. What is the nature of student learning related to disciplinary core ideas, scientific practices, and crosscutting concepts that results from students' engagement in ICE lesson sets? 4. What differences emerge in student engagement and learning outcomes for ICE lessons that incorporate local phenomena or data sets as compared to lessons that do not? 5. What contextual factors (i.e., school context, administrative support, time constraints, etc.) influence teachers' implementation of three-dimensional instruction embedded within ICE lessons?


Project Videos

2019 STEM for All Video Showcase

Title: Integrating Chemistry and Earth Science (ICE)

Presenter(s): Alan Berkowitz, Vonceil Anderson, Bess Caplan, Kevin Garner, & Jonathon Grooms


Engaging Students in Scientific Practices: Evaluating Evidence and Explanation in Secondary Earth and Space Science

This project will develop, implement, test, and revise instructional approaches and materials for high school students that focus on the links between scientific evidence and alternative explanations of phenomena relating to Earth and space education. Students will learn to construct diagrams showing the links between explanatory models of natural phenomena and lines of evidence, and then evaluate the plausibility of various alternative explanations for events.

Award Number: 
2027376
Funding Period: 
Fri, 09/01/2017 to Tue, 08/31/2021
Full Description: 

This project will develop, implement, test, and revise instructional approaches and materials for high school students that focus on the links between scientific evidence and alternative explanations of phenomena relating to Earth and space education. Students will examine alternative explanations for natural phenomena associated with extreme weather events, freshwater resource availability, and related topics in learning how to evaluate scientifically valid lines of evidence and explanation. Students will learn to construct diagrams showing the links between explanatory models of natural phenomena and lines of evidence, and then evaluate the plausibility of various alternative explanations for events. It is expected that engagement in these activities will help students gain proficiency in model-based reasoning, critical thinking, planning and analyzing scientifically valid investigations, constructing plausible explanations, engaging in collaborative argumentation, and critically evaluating scientific information.

This 4-year Design and Development project will examine use of Model-Evidence Link (MEL) diagrams that are intended to help students cognitively construct mental scaffolds that assist their engagement in the practices of critical evaluation, plausibility appraisal, and knowledge construction related to science topics that are considered by some as controversial. Prior research has demonstrated the potential educational outcomes of using MEL diagrams, but this project will extend the previous work by examining an approach where students construct their own MEL diagrams (build-a-MELs, or baMELs). The project will examine the use of both pre-constructed MELs and baMELs for effectiveness in promoting student engagement in scientific reasoning and practices. The project will employ design-based research methodologies in pursuing answers to three research questions: (1) Do baMEL activities tested in multiple high school classroom settings promote critical evaluation, plausibility reappraisal, and  scientifically accurate knowledge construction about controversial Earth and space science topics? (2) How do these additional baMELs differ from pre-constructed MELs in promoting critical evaluation, plausibility reappraisal, and knowledge construction? And (3) To what extent does repeated use of both pre-constructed MELs and baMELs result in student engagement of scientific practices (i.e., asking critical questions, using model-based reasoning, planning and analyzing scientifically valid investigations, constructing plausible explanations, engaging in collaborative argumentation, and critically evaluating scientific information)? The project will engage high school students taking Earth and space classes in selected schools of Georgia, New Jersey, and within Philadelphia. Teacher professional development opportunities associated with the project will include summer institutes, classroom supports, and mentoring sessions.

This project was previously funded under award #1721041.

Mathematical Learning via Architectural Design and Modeling Using E-Rebuild

This project will explore the learning of mathematics through architectural tasks in an online simulation game, E-Rebuild. In the game-based architectural simulation, students will be able to complete tasks such as building and constructing structures while using mathematics and problem solving. The project will examine how to collect data about students' learning from data generated as they play the game, how students learn mathematics using the simulation, and how the simulation can be included in middle school mathematics learning.

Lead Organization(s): 
Award Number: 
1720533
Funding Period: 
Tue, 08/01/2017 to Sat, 07/31/2021
Full Description: 

This project will explore the learning of mathematics through architectural tasks in an online simulation game, E-Rebuild. There is a need to connect mathematics to real world contexts and problems. In the game-based architectural simulation, students will be able to complete tasks such as building and constructing structures while using mathematics and problem solving. The learning platform will be flexible so teachers can customize tasks for their students. The project will examine how to collect data about students' learning from data generated as they play the game. The project will explore how students learn mathematics using the simulation and how the simulation can be included in middle school mathematics learning.

The project includes two major research questions. First, how will the design of a scalable game-based, design-centered learning platform promote coordination and application of math representation for problem solving? Second, how and under what implementation circumstances will using a scalable architectural game-based learning platform improve students multi-stranded mathematical proficiency (i.e., understanding, problem solving and positive disposition)? A key feature of the project is stealth-assessment or data collected and logged as students use the architectural simulation activities that can be used to understand their mathematics learning. The project uses a design-based research approach to gather data from students and teachers that will inform the design of the learning environment. The qualitative and quantitative data will also be used to understand what students are learning as they play the game and how teachers are interacting with their students. The project will include a mixed methods study to compare classrooms using the architectural activities to classrooms that are using typical activities.


Project Videos

2020 STEM for All Video Showcase

Title: E-Rebuild: Scalable Architectural Game for Math Learning

Presenter(s): Fengfeng Ke, Chih-Pu Dai, & Luke West


Fostering Collaborative Computer Science Learning with Intelligent Virtual Companions for Upper Elementary Students (Collaborative Research: Wiebe)

The project will provide the opportunity for upper elementary students to learn computer science and build strong collaboration practices.

Partner Organization(s): 
Award Number: 
1721000
Funding Period: 
Tue, 08/15/2017 to Sat, 07/31/2021
Full Description: 

There is growing recognition that children can, and should, learn computer science. One of the central tenets of computer science is that it is a collaborative discipline, yet children do not start out with an intrinsic ability to collaborate. The project will provide the opportunity for upper elementary students to learn computer science and build strong collaboration practices. Leveraging the promise of virtual learning companions, the project will address three thrusts. First, the project will collect datasets of collaborative learning for computer science in diverse upper elementary school classrooms. Second, the project will design, develop, and iteratively refine its intelligent virtual learning companions, which support dyads of students in a scaffolded computer science learning environment with an interactive online coding tool. Third, the project will generate research findings and evidence about how children collaborate in computer science learning, and how best to support their collaboration with intelligent virtual learning companions. There will be three families of deliverables: learning activities and professional development, an intelligent learning environment with virtual learning companions, and research evidence that furthers the state of scholarship and practice surrounding the collaborative learning of computer science. The project will situate itself in highly diverse elementary schools in two states, Durham County, North Carolina and Alachua County, Florida. This project is supported by the Discovery Research PreK-12 program, which funds research and development of STEM innovations and approaches.

The project addresses the research question, "How can we support upper elementary-school students in computer science learning and collaboration using intelligent virtual learning companions?" The initial dataset will provide a ground-truth measure of students' collaboration approaches to classroom computer science learning tasks through instrumenting computer labs in elementary schools for collecting dialogue and problem-solving activity. The project will collect triangulating qualitative data to better understand impactful classroom dynamics around dyadic learning of computer science. The technical innovation of the project is the way in which student dyads are supported: each pair of children within the elementary school classroom will interact with a dyad of state of-the-art intelligent virtual learning companions. These companions will enhance the classroom experience by adapting in real time to the students' patterns of collaboration and problem solving to provide tailored support specifically for that pair of students. The virtual learning companions will model crucial dimensions of healthy collaboration through their dialogue with one another, including self-explanation, question generation, attributing challenges to the task and not to deficits in each other, and establishing common ground through uptake of ideas. The project will compare outcomes of computer science learning as measured in two ways: individual pre-test to post-test, and quality of collaboratively produced solutions. The project team will measure collaborative practices through dialogue analysis for the target collaboration strategies, as well as interest and self-efficacy for computer science. The project will utilize a multilevel model design to study the effect of the virtual learning companions on student outcomes. Using speech, dialogue transcripts, code artifact analysis, and multimodal analysis of gesture and facial expression, the team will conduct sequential analyses that identify the virtual learning companion interactions that are particularly beneficial for students, and focus our development efforts on expanding and refining those interactions. They will also identify the affordances that students did not engage with and determine whether to eliminate or re-cast them. The analytics of collaborative process data will once again be augmented with qualitative classroom data from field notes, focus groups, and semi-structured interviews with students and teachers. The themes that emerge will guide subsequent refinement of the environment and learning activities.

Exploring the Potential of Tablets as Early Math Resources for Urban Kindergarteners in Schools and Homes

This project will examine the impact on mathematics learning of an initiative to provide kindergartners in an urban school district with personal tablet devices that include free, widely available digital mathematics resources. The research questions examine how teachers use table-based mathematics resources during instruction, how caregivers and children engage with table-based mathematics resources, and how the resources then relate to kindergartners mathematics learning.

Lead Organization(s): 
Award Number: 
1744202
Funding Period: 
Tue, 08/01/2017 to Tue, 07/31/2018
Full Description: 

This project will examine the impact on mathematics learning of an initiative to provide kindergartners in an urban school district with personal tablet devices that include free, widely available digital mathematics resources. An important question for schools as tablet devices become more accessible is how to effectively use them in primary grades, especially kindergarten. In addition, since the devices are portable, how children use the resources such as games for mathematics learning at home is also important to understand. This project is set in a high-needs school district with a large number of low-income children. The project provides an opportunity to learn about the potential role of tables and digital resources in early grades through the analysis of assessment data, user analytic data documenting how the resources were used, and survey data from teachers and families.

Most studies of digital learning resources have been small-scale or focused on engagement. This study offers the opportunity to investigate the relationship between the use of these resources and learning outcomes using a quasi-experimental design. The research questions examine how teachers use table-based mathematics resources during instruction, how caregivers and children engage with table-based mathematics resources and how the resources then relate to kindergartners mathematics learning. Assessments of students' learning will focus on number, geometry and measurement concepts. The learner analytic data from the tablets will document the use of the resources on the tablets. Surveys and demographic data will also be collected to document how the tablets were used. Results of the study should inform implementation of tablet use by schools with particular attention to how they are used across in-class and at-home settings.

Project Accelerate: University-High School AP Physics Partnerships

Project Accelerate blends the supportive structures of a student's home school, a rigorous online course designed specifically with the needs of under-served populations in mind, and hands-on laboratory experiences, to make AP Physics accessible to under-served students. The project could potentially lead to the success of motivated but under-served students who attend schools where the opportunity to engage in a rigorous STEM curriculum is not available.

Lead Organization(s): 
Award Number: 
1720914
Funding Period: 
Tue, 08/01/2017 to Fri, 07/31/2020
Full Description: 

Project Accelerate brings AP Physics 1 and, eventually, AP Physics 2 to students attending schools that do not offer AP Physics. The project will enable 249 students (mostly under-served, i.e., economically disadvantaged, ethnic minorities and racial minorities) to enroll in AP Physics - the students would otherwise not have access. These students either prepare for the AP Physics 1 exam by completing a highly interactive, conceptually rich, rigorous online course, complete with virtual lab experiments, or participate in an accredited AP course that also includes weekly hands-on labs. In this project, the model will be tested and perfected with more students and expanded to AP Physics 2. Further, model replication will be tested at an additional site, beyond the two pilot sites. In the first pilot year in Massachusetts at Boston University, results indicated that students fully engaged in Project Accelerate are (1) at least as well prepared as peer groups in traditional classrooms to succeed on the AP Physics 1 exam and (2) more inclined to engage in additional STEM programs and to pursue STEM fields and programs than they were prior to participating. In the second year of the pilot study, Project Accelerate doubled in size and expanded in partnership with West Virginia University. From lessons learned in the pilot years, key changes are being made, which are expected to increase success. Project Accelerate provides a potential solution to a significant national problem of too few under-served young people having access to high quality physics education, often resulting in these students being ill prepared to enter STEM careers and programs in college. Project Accelerate is a scalable model to empower these students to achieve STEM success, replicable at sites across the country (not only in physics, but potentially across fourteen AP subjects). The project could potentially lead to the success of tens of thousands of motivated but under-served students who attend schools where the opportunity to engage in a rigorous STEM curriculum is not available.

Project Accelerate blends the supportive structures of a student's home school, a private online course designed specifically with the needs of under-served populations in mind, and hands-on laboratory experiences, to make AP Physics accessible to under-served students. The goals of the project are: 1) have an additional 249 students, over three years, complete the College Board-accredited AP Physics 1 course or the AP Physics 1 Preparatory course; 2) add an additional replication site, with a total of three universities participating by the end of the project; 3) develop formal protocols so Project Accelerate can be replicated easily and with fidelity at sites across the nation; 4) develop formal protocols so the project can be self-sustaining at a reasonable cost (about $500 per student participant); 5) build an AP Physics 2 course, giving students who come through AP Physics 1 a second year of rigorous experience to help further prepare them for college and career success; 6) create additional rich interactive content, such as simulations and video-based experiments, to add to what is already in the AP Physics 1 prep course and to build the AP Physics 2 prep course - the key is to actively engage students with the material and include scaffolding to support the targeted population; 7) carry out qualitative and quantitative education research, identifying features of the program that work for the target population, as well as identifying areas for improvement. This project will support the growing body of research on the effectiveness of online and blended (combining online and in-person components) courses, and investigate the use of such courses with under-represented high school students.

Identifying Effective Instructional Practices that Foster the Development of Algebraic Thinking in Elementary School

This project seeks to identify teaching practices that can be linked to students' early algebra learning in grades three, four and five. The goal of the project is to use assessment data and videos of classroom teaching in order to create a tool that can be used to document effective instructional practices. This observation tool can then be used to support teacher professional development in early algebra and research about how teachers' actions can be linked to students' learning.

Lead Organization(s): 
Award Number: 
1721192
Funding Period: 
Thu, 06/01/2017 to Mon, 05/31/2021
Full Description: 

There is a critical need to better prepare all students for learning algebra. Part of this preparation involves developing a strong foundation for algebra in the elementary grades by building on students' informal intuitions about patterns, relationships and structure into more formalized ways of mathematical thinking. This project seeks to identify teaching practices that can be linked to students' early algebra learning in grades three, four and five. The goal of the project is to use assessment data and videos of classroom teaching in order to create a tool that can be used to document effective instructional practices. This observation tool can then be used to support teacher professional development in early algebra and research about how teachers' actions can be linked to students' learning. The project is unique in its work to link an early algebra curriculum with understanding of teachers' practices in implementing that curriculum and students' learning of mathematics.

The project aims to address two research questions. First, what profiles of instructional practice are associated with greater student performance in early algebra? Second, to what extent do these profiles of effective instructional practices vary by grade level? The primary product of the work is an early algebra observation protocol that will capture non-domain and non-grade level specific practices of effective teaching in combination with practices specific to early algebra. Videos of early algebra classrooms will be used to design the observation protocol, which in turn, will then be used along with student assessment data to identify profiles of instructional practices associated with students' learning. Multiple phases of testing and revision will be used to create the observation protocol. The observation protocol will also generate profiles of teacher practices that can be used to describe different models for effectively teaching early algebra. The project will also examine implications of their work for teacher preparation and professional development.

Integration of Engineering Design and Life Science: Investigating the Influence of an Intervention on Student Interest and Motivation in STEM Fields

This project will investigate the integration of engineering design, practices, and thinking into middle school life science curriculum while providing opportunities for students to foster knowledge of and increase interest in life and biosciences. The project will specifically respond to the need to create, implement, and evaluate a model intervention that will advance the knowledge base for establishing and retaining underrepresented minorities in STEM fields.

Lead Organization(s): 
Award Number: 
1721141
Funding Period: 
Fri, 09/01/2017 to Tue, 08/31/2021
Full Description: 

This project will investigate the integration of engineering design, practices, and thinking into middle school life science curriculum while providing opportunities for students to foster knowledge of and increase interest in life and biosciences. The project will specifically respond to the need to create, implement, and evaluate a model intervention that will advance the knowledge base for establishing and retaining underrepresented minorities in STEM fields. Specifically, the project will partner with middle school science teachers from two local school corporations, STEM university faculty members and undergraduate engineering students, and university-based outreach coordinators from a minorities engineering program, the office of future engineers, and women in engineering program. Through this combined effort, both school corporations that serve underserved, culturally diverse, and socioeconomically disadvantaged students in rural communities; will have broad-based support for engaging 36 teachers and 3000 students in integrated life science with engineering design.

The project will employ a mixed methods research design incorporating both qualitative and quantitative approaches for data collection and analyses. The research team will conduct quantitative analyses by using Hierarchical Linear Modeling to determine the extent to which integrating life science with engineering design and thinking impact student learning of life science concepts and interest in life and biosciences. Qualitative approaches, including discourse analysis, will be used to delve deeper into student learning of the targeted life science concepts. Through this research, the project will advance evidence-based understanding of learning, enhance the theoretical models of student life science learning, and merge and extend the successes of previous studies by using the faculty expertise in effective approaches in engineering integration in K-12 science classrooms. Specifically, concept assessments, interest surveys, recordings of classroom discourse, student artifacts (e.g., design reports), interviews, and classroom observations will be used as data sources. Outcomes from the project will advance the knowledge base for establishing and retaining underrepresented minorities in STEM fields. The life STEM focused design tasks will be disseminated through an online peer-reviewed digital library available for use across the U.S. and beyond. Along with the design-based tasks on this website; results from the intervention model will be disseminated through electronic and print media to inform researchers, educators, administrators, and policy makers who play critical roles in enhancing student learning of and interest in STEM, about pathways to broadening participation in STEM.

Pages

Subscribe to Mixed Methods