Mixed Methods

Enhancing Teacher and Student Understanding of Engineering in K-5 Bilingual Programs

This mixed-method exploratory study will examine how bilingual teachers working in elementary schools in Massachusetts and Puerto Rico understand the role and skills of engineers in society. In turn, it will examine how teachers adapt existing engineering lessons so that those activities and concepts are more culturally and linguistically accessible to their students.

Lead Organization(s): 
Award Number: 
1814258
Funding Period: 
Mon, 10/01/2018 to Thu, 09/30/2021
Full Description: 

Engineering is part of everyone's local community and daily activities yet opportunities to learn about engineering are often absent from elementary school classrooms. Further, little is known about how teachers' and students' conceptions of engineering relate to aspects of their local community such as language and culture. Knowing more about this is important because students' perceptions of mismatch between their personal culture and the engineering field contributes to the continued underrepresentation of minorities in the profession. This mixed-method exploratory study will examine how bilingual teachers working in elementary schools in Massachusetts and Puerto Rico understand the role and skills of engineers in society. In turn, it will examine how teachers adapt existing engineering lessons so that those activities and concepts are more culturally and linguistically accessible to their students.

Consistent with the aims of the DRK-12 program, this project will advance understanding of how engineering education materials can be adapted to the characteristics of teachers, students, and the communities that they reside in. Further, its focus on bilingual classrooms will bring new perspectives to characterizations of the engineering field and its role in different cultures and societies. Over a three-year period, the team will investigate these issues by collecting data from 24 teachers (12 from each location). Data will be collected via surveys, interviews, discussion of instructional examples, videos of teachers' classroom instruction and analysis of artifacts such as teachers' lesson plans. Teachers will collaborate and function as a professional co-learning community called instructional rounds by participating and providing feedback synchronously in face-to-face settings and via the use of digital apps. Project findings can lead to teaching guidelines, practices, and briefs that inform efforts to successfully integrate bilingual engineering curriculum at the elementary grades. This work also has the potential to create professional development models of success for K-5 teachers in bilingual programs and enhance engineering teaching strategies and methods at these early grade levels.

Engaging High School Students in Computer Science with Co-Creative Learning Companions (Collaborative Research: Magerko)

This research investigates how state-of-the-art creative and pedagogical agents can improve students' learning, attitudes, and engagement with computer science. The project will be conducted in high school classrooms using EarSketch, an online computer science learning environments that engages learners in making music with JavaScript or Python code.

Award Number: 
1814083
Funding Period: 
Sat, 09/15/2018 to Wed, 08/31/2022
Full Description: 
This research investigates how state-of-the-art creative and pedagogical agents can improve students' learning, attitudes, and engagement with computer science. The project will be conducted in high school classrooms using EarSketch, an online computer science learning environments that engages over 160,000 learners worldwide in making music with JavaScript or Python code. The researchers will build the first co-creative learning companion, Cai, that will scaffold students with pedagogical strategies that include making use of learner code to illustrate abstraction and modularity, suggesting new code to scaffold new concepts, providing help and hints, and explaining its decisions. This work will directly address the national need to develop computing literacy as a core STEM skill.
 
The proposed work brings together an experienced interdisciplinary team to investigate the hypothesis that adding a co-creative learning companion to an expressive computer science learning environment will improve students' computer science learning (as measured by code sophistication and concept knowledge), positive attitudes towards computing (self-efficacy and motivation), and engagement (focused attention and involvement during learning). The iterative design and development of the co-creative learning companion will be based on studies of human collaboration in EarSketch classrooms, the findings in the co-creative literature and virtual agents research, and the researchers' observations of EarSketch use in classrooms. This work will address the following research questions: 1) What are the foundational pedagogical moves that a co-creative learning companion for expressive programming should perform?; 2) What educational strategies for a co-creative learning companion most effectively scaffold learning, favorable attitudes toward computing, and engagement?; and 3) In what ways does a co-creative learning companion in EarSketch increase computer science learning, engagement, and positive attitudes toward computer science when deployed within the sociocultural context of a high school classroom? The proposed research has the potential to transform our understanding of how to support student learning in and broaden participation through expressive computing environments.

Science, Technology, Engineering and Mathematics Teaching in Rural Areas Using Cultural Knowledge Systems

This project will collaborate with Indigenous communities to create educational resources serving Inupiaq middle school students and their teachers. The Cultural Connections Process Model (CCPM) will formalize, implement, and test a process model for community-engaged educational resource development for Indigenous populations. The project will contribute to a greater understanding of effective natural science teaching and science career recruitment of minority students.

Award Number: 
1812888
Funding Period: 
Sat, 09/01/2018 to Tue, 08/31/2021
Full Description: 

The Cultural Connections Process Model (CCPM) will formalize, implement, and test a process model for community-engaged educational resource development for Indigenous populations. The project will collaborate with Indigenous communities to create educational resources serving Inupiaq middle school students and their teachers. Research activities take place in Northwest Alaska. Senior personnel will travel to rural communities to collaborate with and support participants. The visits demonstrate University of Alaska Fairbanks's commitment to support pathways toward STEM careers, community engagement in research, science teacher recruitment and preparation, and STEM career awareness for Indigenous and rural pre-college students. Pre-service teachers who access to the resources and findings from this project will be better prepared to teach STEM to Native students and other minorities and may be more willing to continue careers as science educators teaching in settings with Indigenous students. The project will contribute to a greater understanding of effective natural science teaching and science career recruitment of minority students. The project's participants and the pre-college students they teach will be part of the pipeline into science careers for underrepresented Native students in Arctic communities. The project will build on partnerships outside of Alaska serving other Indigenous populations and will expand outreach associated with NSF's polar science investments.

CCPM will build on cultural knowledge systems and NSF polar research investments to address science themes relevant to Inupiat people, who have inhabited the region for thousands of years. An Inupiaq scholar will conduct project research and guide collaboration between Indigenous participants and science researchers using the Inupiaq research methodology known as Katimarugut (meaning "we are meeting"). The project research and development will engage 450 students in grades 6-8 and serves 450 students (92% Indigenous) and 11 teachers in the remote Arctic. There are two broad research hypotheses. The first is that the project will build knowledge concerning STEM research practices by accessing STEM understandings and methodologies embedded in Indigenous knowledge systems; engaging Indigenous communities in project development of curricular resources; and bringing Arctic science research aligned with Indigenous priorities into underserved classrooms. The second is that classroom implementation of resources developed using the CCPM will improve student attitudes toward and engagement with STEM and increase their understandings of place-based science concepts. Findings from development and testing will form the basis for further development, broader implementation and deeper research to inform policy and practice on STEM education for underrepresented minorities and on rural education.

Design and Development of a K-12 STEM Observation Protocol (Collaborative Research: Roehrig)

This project will design and develop a new K-12 classroom observation protocol for integrated STEM instruction (STEM-OP). The STEM-OP will inform the instruction of integrated STEM in many contexts with the goal of improving integrated STEM education.

Award Number: 
1813342
Funding Period: 
Sat, 09/01/2018 to Wed, 08/31/2022
Full Description: 

This project will design and develop a new K-12 classroom observation protocol for integrated STEM instruction (STEM-OP). The STEM-OP will be developed for use in K-12 STEM settings. While the importance of integrated STEM education is established, there remains disagreement on models and effective approaches for integrated STEM instruction. This issue is confounded by the lack of observation protocols sensitive to integrated STEM teaching and learning to inform research to the effectiveness of new models and strategies. Existing instruments were not developed for use in integrated STEM learning environments. The STEM-OP will be designed to be used effectively by multiple stakeholders in a variety of contexts. Researchers will benefit from having the STEM-OP available for them to carry out research and continue to improve STEM education in a variety of ways. Existing instruments were not developed for use in integrated STEM learning environments.  The STEM-OP and associated training materials will be available for use by other education stakeholders, such as K-12 teachers and district administrators, through a publicly available online platform. In brief, the STEM-OP will inform the instruction of integrated STEM in many contexts with the goal of improving integrated STEM education.

The primary product of this project is the new observation protocol called STEM-OP for K-12 classrooms implementing integrated STEM lessons. The project will use over 500 integrated STEM classroom videos to design the STEM-OP. Using exploratory and confirmatory factor analysis, the STEM-OP will be a valid and reliable instrument for use in a variety of educational contexts. The research will explore the different ways that elementary, middle, and high school science teachers enact integrated STEM instruction. This study will shed light on the nature of STEM instruction in each of these grade bands and provide information building towards an understanding of learning progressions for engineering practices across grade bands. Research exploring how the nature of STEM integration changes from day to day over the course of a unit will provide critical information about the different sequencing and trajectories of STEM units. Examining how integrated STEM instruction unfolds over a full unit of instruction will inform the understanding of integrated STEM practices at both micro- and macro- levels of analysis. The STEM-OP and associated training materials will be available for use by other education stakeholders, such as K-12 teachers and district administrators, through a publicly available, which will be distributed via a publicly available, online platform that includes a training manual and classroom video for practice scoring.

Design and Development of a K-12 STEM Observation Protocol (Collaborative Research: Dare)

This project will design and develop a new K-12 classroom observation protocol for integrated STEM instruction (STEM-OP). The STEM-OP will inform the instruction of integrated STEM in many contexts with the goal of improving integrated STEM education.

Lead Organization(s): 
Award Number: 
1854801
Funding Period: 
Sat, 09/01/2018 to Wed, 08/31/2022
Full Description: 

This project will design and develop a new K-12 classroom observation protocol for integrated STEM instruction (STEM-OP). The STEM-OP will be developed for use in K-12 STEM settings. While the importance of integrated STEM education is established, there remains disagreement on models and effective approaches for integrated STEM instruction. This issue is confounded by the lack of observation protocols sensitive to integrated STEM teaching and learning to inform research to the effectiveness of new models and strategies. Existing instruments were not developed for use in integrated STEM learning environments. The STEM-OP will be designed to be used effectively by multiple stakeholders in a variety of contexts. Researchers will benefit from having the STEM-OP available for them to carry out research and continue to improve STEM education in a variety of ways. Existing instruments were not developed for use in integrated STEM learning environments.  The STEM-OP and associated training materials will be available for use by other education stakeholders, such as K-12 teachers and district administrators, through a publicly available online platform. In brief, the STEM-OP will inform the instruction of integrated STEM in many contexts with the goal of improving integrated STEM education.

The primary product of this project is the new observation protocol called STEM-OP for K-12 classrooms implementing integrated STEM lessons. The project will use over 500 integrated STEM classroom videos to design the STEM-OP. Using exploratory and confirmatory factor analysis, the STEM-OP will be a valid and reliable instrument for use in a variety of educational contexts. The research will explore the different ways that elementary, middle, and high school science teachers enact integrated STEM instruction. This study will shed light on the nature of STEM instruction in each of these grade bands and provide information building towards an understanding of learning progressions for engineering practices across grade bands. Research exploring how the nature of STEM integration changes from day to day over the course of a unit will provide critical information about the different sequencing and trajectories of STEM units. Examining how integrated STEM instruction unfolds over a full unit of instruction will inform the understanding of integrated STEM practices at both micro- and macro- levels of analysis. The STEM-OP and associated training materials will be available for use by other education stakeholders, such as K-12 teachers and district administrators, through a publicly available, which will be distributed via a publicly available, online platform that includes a training manual and classroom video for practice scoring.

Design and Development of a K-12 STEM Observation Protocol (Collaborative Research: Ring-Whalen)

This project will design and develop a new K-12 classroom observation protocol for integrated STEM instruction (STEM-OP). The STEM-OP will inform the instruction of integrated STEM in many contexts with the goal of improving integrated STEM education.

Lead Organization(s): 
Award Number: 
1812794
Funding Period: 
Sat, 09/01/2018 to Wed, 08/31/2022
Full Description: 

This project will design and develop a new K-12 classroom observation protocol for integrated STEM instruction (STEM-OP). The STEM-OP will be developed for use in K-12 STEM settings. While the importance of integrated STEM education is established, there remains disagreement on models and effective approaches for integrated STEM instruction. This issue is confounded by the lack of observation protocols sensitive to integrated STEM teaching and learning to inform research to the effectiveness of new models and strategies. Existing instruments were not developed for use in integrated STEM learning environments. The STEM-OP will be designed to be used effectively by multiple stakeholders in a variety of contexts. Researchers will benefit from having the STEM-OP available for them to carry out research and continue to improve STEM education in a variety of ways. Existing instruments were not developed for use in integrated STEM learning environments.  The STEM-OP and associated training materials will be available for use by other education stakeholders, such as K-12 teachers and district administrators, through a publicly available online platform. In brief, the STEM-OP will inform the instruction of integrated STEM in many contexts with the goal of improving integrated STEM education.

The primary product of this project is the new observation protocol called STEM-OP for K-12 classrooms implementing integrated STEM lessons. The project will use over 500 integrated STEM classroom videos to design the STEM-OP. Using exploratory and confirmatory factor analysis, the STEM-OP will be a valid and reliable instrument for use in a variety of educational contexts. The research will explore the different ways that elementary, middle, and high school science teachers enact integrated STEM instruction. This study will shed light on the nature of STEM instruction in each of these grade bands and provide information building towards an understanding of learning progressions for engineering practices across grade bands. Research exploring how the nature of STEM integration changes from day to day over the course of a unit will provide critical information about the different sequencing and trajectories of STEM units. Examining how integrated STEM instruction unfolds over a full unit of instruction will inform the understanding of integrated STEM practices at both micro- and macro- levels of analysis. The STEM-OP and associated training materials will be available for use by other education stakeholders, such as K-12 teachers and district administrators, through a publicly available, which will be distributed via a publicly available, online platform that includes a training manual and classroom video for practice scoring.

Design and Development of Transmedia Narrative-based Curricula to Engage Children in Scientific Thinking and Engineering Design (Collaborative Research: Ellis)

This project will address the need for engineering resources by applying an innovative pedagogy called Imaginative Education (IE) to create middle school engineering curricula. In IE, developmentally appropriate narratives are used to design learning environments that help learners engage with content and organize their knowledge productively. This project will combine IE with transmedia storytelling.

Lead Organization(s): 
Award Number: 
1814033
Funding Period: 
Sun, 07/15/2018 to Thu, 06/30/2022
Full Description: 

Engineering is an important component of the Next Generation Science Standards (NGSS). However, resources for supporting teachers in implementing these standards are scarce. This project will address the need for resources by applying an innovative pedagogy called Imaginative Education (IE) to create middle school engineering curricula. In IE, developmentally appropriate narratives are used to design learning environments that help learners engage with content and organize their knowledge productively. To fully exploit the potential of this pedagogy, this project will combine IE with transmedia storytelling. In transmedia storytelling, different elements of a narrative are spread across a variety of formats (such as books, websites, new articles, videos and other media) in a way that creates a coordinated experience for the user. Once created, the curricula will be implemented in classrooms to research its impact on (1) increasing learners' capacities to engage in both innovative and direct application of engineering concepts, and (2) improving learners' science, technology, engineering, and mathematics (STEM) identity. 

This research will be led by Smith College and Springfield Technical Community College in collaboration with Springfield (MA) Public Schools (SPS). Additional expertise in evaluating the findings will be provided by the Collaborative for Educational Services and an external advisory board of leaders in STEM education and transmedia storytelling. The project will result in the development of a transmedia learning environment that includes two NGSS-aligned, interdisciplinary engineering units and seven lessons that integrate science and engineering. The research study will be implemented in four phases in eight SPS middle schools. Approximately 900 students will participate each year. In Phase 1, the project team will collaborate with SPS teachers to create engineering units, lessons, and standards-based achievement measures. In Phase 2, teachers in the treatment group will participate in professional development (PD) workshops covering IE, transmedia learning environments, structure of the curriculum, and connections to NGSS. In Phase 3 the curricula will be implemented in treatment classrooms and both treatment and control group students will be assessed. In Phase 4, testing and assessment will continue in SPS schools and will be expanded to rural and suburban classrooms. Teachers in these classrooms will use online multimedia PD that will ensure scalability and mirrors the structure and content of in-person PD. Data analysis will provide evidence of whether this imaginative and transmedia educational approach improves students' capacities for using engineering concepts and enhances their STEM identity.


Project Videos

2019 STEM for All Video Showcase

Title: Transforming Engineering Education for Middle School (TEEMS)

Presenter(s): Beth McGinnis-Cavanaugh, Sonia Ellis, & Crystal Ford


Design and Development of Transmedia Narrative-based Curricula to Engage Children in Scientific Thinking and Engineering Design (Collaborative Research: McGinnis-Cavanaugh)

This project will address the need for engineering resources by applying an innovative pedagogy called Imaginative Education (IE) to create middle school engineering curricula. In IE, developmentally appropriate narratives are used to design learning environments that help learners engage with content and organize their knowledge productively. This project will combine IE with transmedia storytelling.

Partner Organization(s): 
Award Number: 
1813572
Funding Period: 
Sun, 07/15/2018 to Thu, 06/30/2022
Project Evaluator: 
Collaborative for Educational Services (CES)
Full Description: 

Engineering is an important component of the Next Generation Science Standards (NGSS). However, resources for supporting teachers in implementing these standards are scarce. This project will address the need for resources by applying an innovative pedagogy called Imaginative Education (IE) to create middle school engineering curricula. In IE, developmentally appropriate narratives are used to design learning environments that help learners engage with content and organize their knowledge productively. To fully exploit the potential of this pedagogy, this project will combine IE with transmedia storytelling. In transmedia storytelling, different elements of a narrative are spread across a variety of formats (such as books, websites, new articles, videos and other media) in a way that creates a coordinated experience for the user. Once created, the curricula will be implemented in classrooms to research its impact on (1) increasing learners' capacities to engage in both innovative and direct application of engineering concepts, and (2) improving learners' science, technology, engineering, and mathematics (STEM) identity. 

This research will be led by Smith College and Springfield Technical Community College in collaboration with Springfield (MA) Public Schools (SPS). Additional expertise in evaluating the findings will be provided by the Collaborative for Educational Services and an external advisory board of leaders in STEM education and transmedia storytelling. The project will result in the development of a transmedia learning environment that includes two NGSS-aligned, interdisciplinary engineering units and seven lessons that integrate science and engineering. The research study will be implemented in four phases in eight SPS middle schools. Approximately 900 students will participate each year. In Phase 1, the project team will collaborate with SPS teachers to create engineering units, lessons, and standards-based achievement measures. In Phase 2, teachers in the treatment group will participate in professional development (PD) workshops covering IE, transmedia learning environments, structure of the curriculum, and connections to NGSS. In Phase 3 the curricula will be implemented in treatment classrooms and both treatment and control group students will be assessed. In Phase 4, testing and assessment will continue in SPS schools and will be expanded to rural and suburban classrooms. Teachers in these classrooms will use online multimedia PD that will ensure scalability and mirrors the structure and content of in-person PD. Data analysis will provide evidence of whether this imaginative and transmedia educational approach improves students' capacities for using engineering concepts and enhances their STEM identity.


Project Videos

2019 STEM for All Video Showcase

Title: Transforming Engineering Education for Middle School (TEEMS)

Presenter(s): Beth McGinnis-Cavanaugh, Sonia Ellis, & Crystal Ford


Extending and Investigating the Impact of the High School Model-based Educational Resource (Collaborative Research: Passmore)

This project builds on a line of work that has developed and studied the Model Based Educational Resource (MBER), a year-long curriculum for high school biology. The project will generate rigorous causal evidence on how this approach to biology teaching and learning can support student learning, and foundational information on how to support high school teachers in improving their teaching. It will also provide resources to expand and update MBER to reflect the changing high school science landscape by integrating Earth Science standards into the year long sequence.

Partner Organization(s): 
Award Number: 
1814263
Funding Period: 
Wed, 08/01/2018 to Sun, 07/31/2022
Full Description: 

This project builds on a line of work that has developed and studied the Model Based Educational Resource (MBER), a year-long curriculum for high school biology. In classrooms using MBER, modeling serves as an anchoring practice that keeps the inquiry tied to the goal of making sense of the world, helping teachers to engage their students in a range of cognitive and social activities that lead to deep understanding of scientific ideas. This project will generate rigorous causal evidence on how this approach to biology teaching and learning can support student learning, and foundational information on how to support high school teachers in improving their teaching. This funding will also provide resources to expand and update MBER to reflect the changing high school science landscape by integrating Earth Science standards into the year long sequence. The study will address the general research question: What is the impact of the Model Based Educational Resource (MBER) on high school students' science achievement, and what factors influence that impact? In addition to generating important research findings, the materials revised and studied in this project will be open-source and freely available to teachers and schools.

This study addresses a significant gap in the research on next generation curriculum materials. While there is emerging agreement about the importance of instructional materials in supporting teachers in effectively engaging students in the practices of science, there is very little empirical evidence to support such claims. The goal of this project is to study the impact of the MBER program through a cluster randomized trial and expand the promise of efficacy and feasibility established in previous work. This study will be able to make causal claims by using an experimental design in which 32 high school teachers serve as their own controls, and by using multi-level modeling in the analysis. This study will advance the field's knowledge about the impact of innovative materials on student learning, measured by both project-level assessments and the state science test. Exploratory research questions will examine a) how using the MBER program develops teachers' vision of the Next Generation Science Standards, b) how student learning is mediated by the fidelity of implementation of the materials, c) how teachers interact with materials designed to be modified for their classroom context, and d) to what extent the MBER materials provide equitable opportunities to learn and close achievement gaps.

Strengthening Data Literacy Across the Curriculum

This project will develop a set of statistics learning materials, with data visualization tools and an applied social science focus, to design applied data investigations addressing real-world socioeconomic questions with large-scale social science data. This project is designed to promote statistical understandings and interest in quantitative data analysis among high school students and engage students with content that resonates with their interests.

Award Number: 
1813956
Funding Period: 
Sun, 07/01/2018 to Wed, 06/30/2021
Full Description: 

The Strengthening Data Literacy across the Curriculum (SDLC) project seeks to significantly enhance the learning and teaching of Science, Technology, Engineering, and Mathematics (STEM) high school students and teachers through the development of resources, models, and tools. This project is designed to promote statistical understandings and interest in quantitative data analysis among high school students. The project will target students outside mathematics and statistics classes who seldom have opportunities formally make sense of large-scale quantitative data. The population for the initial study will be humanities/social studies and mathematics/statistics high school teachers and their classes. The focus on social justice themes are intended to engage students with content that resonates with their interests. This strategy has the potential to demonstrate ways to provide rich, meaningful statistical instruction to a population that seldom has the opportunity for such learning. By capturing students' imagination and interest with social justice themes, this project has the potential of high impact in today's society where understanding and preparing statistical reports are becoming more critical to the general populace.

This project will build on prior theory and research to develop a new set of statistics learning materials, with data visualization tools and an applied social science focus to design three 2-week applied data investigations (self-contained modules) addressing real-world socioeconomic questions with large-scale social science data. The modules will be aligned with the high school Common Core State Standards for Mathematics and key statistical content for college students. The purpose of the study is to strengthen existing theories of how to design classroom learning materials to support two primary sets of outcomes for high school students, particularly among those historically underrepresented in STEM fields: 1) stronger understandings of important statistics concepts and data analysis practices, and 2) interest in statistics and working with data.  The modules will engage students in a four-step investigative process where they will (1) formulate questions that can be answered with data; (2) design and implement a plan to assemble appropriate data; (3) use numerical and graphical methods to explore the data; and (4) summarize conclusions relating back to the original questions and citing relevant components of the analysis that support their interpretation and acknowledging other interpretations.

The project will employ a Design-Based Implementation Research (DBIR) design using both quantitative and qualitative data to determine results of targeted outcomes (noted above) as well track whether there is any evidence to support the conjectures that key module components directly impact targeted student outcomes. Starting with a well-defined, preliminary conceptual framework for the study, the project team will conduct four cycles of iterative design and testing of the proposed SDLC modules over two academic years, with each cycle occurring during a fall or spring semester.

Pages

Subscribe to Mixed Methods