Engineering

Improving Evaluations of STEM Programs: An Empirical Investigation of Key Design Parameters

This study seeks to further understanding of the STEM learning environment by 1) examining the extent to which mathematics and science achievement varies across students, teachers, schools, and districts, and 2) examining the extent to which student, teacher, school, and district characteristics that are found in state administrative databases can be used to explain this variation at each level.

Lead Organization(s): 
Award Number: 
2000388
Funding Period: 
Mon, 06/15/2020 to Wed, 05/31/2023
Full Description: 

To improve science, technology, engineering, and mathematics (STEM) outcomes in K-12 classrooms, it is critical to understand the landscape of the STEM learning environment. However, the STEM learning environment is complex. Students are nested within teachers, and teachers are nested within schools (which in turn are nested within districts), which implies a multilevel structure. To date, most empirical research that uses multilevel modeling to examine the role of higher levels on variation in student outcomes does not examine the teacher level. The reason is that for many states, data linkages between students and teachers have been difficult to achieve. However, in the last five years, this situation has improved in many states, which makes this work now possible. This study seeks to further understanding of the STEM learning environment by 1) examining the extent to which mathematics and science achievement varies across students, teachers, schools, and districts and 2) examining the extent to which student, teacher, school, and district characteristics that are found in state administrative databases can be used to explain this variation at each level. This work will support advances in research and evaluation methodologies that will enable researchers to design more rigorous and comprehensive evaluations of STEM interventions and improve the accuracy of statistical power calculations. Broad dissemination of the resulting tools and techniques will provide access through freely available websites, and workshops and training opportunities to build capacity in the field for STEM researchers to design cluster randomized trials (CRTs) to answer questions beyond what works to for whom and under what conditions.

This project will contribute to 1) describing and explaining the landscape of the STEM learning environment, an environment which includes students, teachers, and schools, and 2) applying this empirical information in the design of STEM intervention studies to enable researchers to extend beyond the usual questions about if the intervention works and for which types of students or schools. By adding teacher level variables, this analysis would account for key teacher characteristics that may moderate the treatment effect. The research will also increase the efficiency in the design of CRTs of STEM interventions. Specifically, the findings from this study will improve the internal validity and cost-efficiency of evaluations of STEM interventions by increasing the accuracy of estimates for the full range of parameters needed to conduct power analyses, particularly when the teacher level is included. The high cost associated with CRTs makes it critical to plan accurate trials that do not overestimate the required sample size, and hence cost more than necessary, or underestimate the sample size and thereby reduce the potential to generate high-quality evidence of program effectiveness. Including the teacher level in intervention studies, a critical level in the delivery of any intervention, will allow for more testing of teacher characteristics that may moderate intervention effects.

Anchoring High School Students in Real-Life Issues that Integrate STEM Content and Literacy

Through the integration of STEM content and literacy, this project will study the ways teachers implement project practices integrating literacy activities into STEM learning. Teachers will facilitate instruction using scenarios that present students with everyday, STEM-related issues, presented as scenarios, that they read and write about. After reading and engaging with math and science content, students will write a source-based argument in which they state a claim, support the claim with evidence from the texts, and explain the multiple perspectives on the issue.

Lead Organization(s): 
Award Number: 
2010312
Funding Period: 
Sat, 08/15/2020 to Sun, 07/31/2022
Full Description: 

The STEM Literacy Project sets out to support student learning through developing teacher expertise in collaborative integration of STEM in student writing and literacy skills development. Facilitated by teachers, students will read, discuss, and then write about real-world STEM scenarios, such as water quality or health. The project will build on and research a professional development program first developed through a state-supported literacy program for middle and high school science and math teachers to improve literacy-integrated instruction. The goals of this project include the following: (1) Create a community of practice that recognizes high school teachers as content experts; (2) Implement high quality professional development for teachers on STEM/Literacy integration; (3) Develop assessments based on STEM and literacy standards that inform instruction; and (4) Conduct rigorous research to understand the impact of the professional development. The program is aligned with state and national standards for college and career readiness. Project resources will be widely shared through a regularly updated project website (stemliteracyproject.org), conference presentations, and publications reaching researchers, developers, and educators. These resources will include scenario-based assessment tools and instructional materials.

Through the integration of STEM content and literacy, the project will study the ways teachers implement project practices integrating literacy activities into STEM learning. Teachers will facilitate instruction using scenarios that present students with everyday, STEM-related issues, presented as scenarios, that they read and write about. After reading and engaging with math and science content, students will write a source-based argument in which they state a claim, support the claim with evidence from the texts, and explain the multiple perspectives on the issue. These scenarios provide students with agency as they craft an argument for an audience, such as presenting to a city council, a school board, or another group of stakeholders. Project research will use a mixed methods design. Based on the work completed through the initial designs and development of scenario-based assessments, rubrics, and scoring processes, the project will study the impact on instruction and student learning. Using a triangulation design convergence model, findings will be compared and contrasted in order for the data to inform one another and lead to further interpretation of the data. project will analyze the features of STEM content learning after program-related instruction. Data collected will include pre-post student scenario-based writing; pre-post interviews of up to 40 students each year; pre-post teacher interviews; and teacher-created scenario-based assessments and supporting instructional materials. Student learning reflected in the assessments paired with student and teacher interview responses will provide a deeper understanding of this approach of integrating STEM and literacy. The use of discourse analysis methods will allow growth in content learning to be measured through language use. Project research will build knowledge in the field concerning how participation in teacher professional development integrating STEM content in literacy practices impacts teacher practices and student learning.

Design Talks: Building Community with Elementary Engineering (Collaborative Research: Watkins)

This project explores how classroom conversations can engage children in making sense of the problems that they are addressing and foregrounding ethics while making design decisions. To provide children with opportunities to engage in rich classroom conversations, the project team uses a community-based engineering curricular approach, where students address problems that affect their local school communities.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
2010237
Funding Period: 
Sat, 08/01/2020 to Mon, 07/31/2023
Full Description: 

Inclusion of engineering design activities in elementary classrooms has become increasingly common, and teachers are becoming more comfortable with the basics of teaching engineering. There is now a need and an opportunity to understand different approaches teachers can take to support students to deepen their understanding of engineering design content knowledge and engineering practices. While many existing approaches to preK-12 engineering education emphasize problem solving and the development of engineering solutions, this project also explores how classroom conversations can engage children in making sense of the problems that they are addressing and foregrounding ethics while making design decisions. To provide children with opportunities to engage in rich classroom conversations, the project team uses a community-based engineering curricular approach, where students address problems that affect their local school communities. The discussion-focused, community-based engineering curricular approach has promise in providing opportunities for children to practice sense-making and decision-making skills and also develop a perspective of care as central to engineering design work.

To accomplish this project, the researchers extend an ongoing partnership with two elementary teachers to implement the discussion-rich community-based engineering curricular approach and collect video-recordings of the elementary students' engineering design conversations. The videos will be analyzed using discourse analysis to generate evidence-based theory on the characteristics and dynamics of classroom talk that support elementary students' knowledge construction in engineering design contexts, as well as theory on how teachers prompt them and elicit meaningful participation from all students. By providing additional resources and an intellectual framework for investigating and prompting meaningful disciplinary discourse in engineering design, the project will support the two partner teachers to apprentice eight of their colleagues over three years into the work of community-based engineering and design talk. This collaboration will develop resources that will support teachers and students to engage in more caring, ethical discourse around design. Specifically, the project team will create an online video library of design talk resources for grade 1-6 classroom teachers. The Design Talk website will enable elementary teachers to see distinctly different kinds of classroom conversations that make elementary engineering a site for students not just to build products, but also to build knowledge.

Design Talks: Building Community with Elementary Engineering (Collaborative Research: Wendell)

This project explores how classroom conversations can engage children in making sense of the problems that they are addressing and foregrounding ethics while making design decisions. To provide children with opportunities to engage in rich classroom conversations, the project team uses a community-based engineering curricular approach, where students address problems that affect their local school communities.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
2010139
Funding Period: 
Sat, 08/01/2020 to Mon, 07/31/2023
Full Description: 

Inclusion of engineering design activities in elementary classrooms has become increasingly common, and teachers are becoming more comfortable with the basics of teaching engineering. There is now a need and an opportunity to understand different approaches teachers can take to support students to deepen their understanding of engineering design content knowledge and engineering practices. While many existing approaches to preK-12 engineering education emphasize problem solving and the development of engineering solutions, this project also explores how classroom conversations can engage children in making sense of the problems that they are addressing and foregrounding ethics while making design decisions. To provide children with opportunities to engage in rich classroom conversations, the project team uses a community-based engineering curricular approach, where students address problems that affect their local school communities. The discussion-focused, community-based engineering curricular approach has promise in providing opportunities for children to practice sense-making and decision-making skills and also develop a perspective of care as central to engineering design work.

To accomplish this project, the researchers extend an ongoing partnership with two elementary teachers to implement the discussion-rich community-based engineering curricular approach and collect video-recordings of the elementary students' engineering design conversations. The videos will be analyzed using discourse analysis to generate evidence-based theory on the characteristics and dynamics of classroom talk that support elementary students' knowledge construction in engineering design contexts, as well as theory on how teachers prompt them and elicit meaningful participation from all students. By providing additional resources and an intellectual framework for investigating and prompting meaningful disciplinary discourse in engineering design, the project will support the two partner teachers to apprentice eight of their colleagues over three years into the work of community-based engineering and design talk. This collaboration will develop resources that will support teachers and students to engage in more caring, ethical discourse around design. Specifically, the project team will create an online video library of design talk resources for grade 1-6 classroom teachers. The Design Talk website will enable elementary teachers to see distinctly different kinds of classroom conversations that make elementary engineering a site for students not just to build products, but also to build knowledge.

Exploring Changes in Teachers' Engineering Design Self-Efficacy and Practice through Collaborative and Culturally Relevant Professional Development

In this project, investigators from the University of North Dakota develop, evaluate, and implement an on-going, collaborative professional development program designed to support teachers in teaching engineering design to 5th-8th grade students in rural and Native American communities.

Lead Organization(s): 
Award Number: 
2010169
Funding Period: 
Fri, 01/01/2021 to Sun, 12/31/2023
Full Description: 

Promoting diverse, inclusive and equitable participation in engineering design education at the elementary and middle school levels is important for a number of reasons. In addition to benefits of a diverse STEM workforce to industry and the economy, youth are better able to make informed decisions about pursuing STEM degrees and STEM career pathways and youth are able to develop critical thinking and problem solving skills that allow them to be creative and innovative problem solvers. However, for youth to participate in inclusive and equitable engineering design experiences in elementary and middle schools settings, teachers need opportunities to develop engineering content knowledge, pedagogical content knowledge, and strategies for culturally-relevant teaching. In this project, investigators from the University of North Dakota develop, evaluate, and implement an on-going, collaborative professional development program designed to support teachers in teaching engineering design to 5th-8th grade students in rural and Native American communities.

The project advances the understanding of teacher training in K-12 engineering education and more specifically culturally-relevant engineering design education for 5th-8th grade students. The program design is guided by principles from Bandura's Social Learning Theory, Gladson-Billing's culturally-relevant teaching, and Gay's cultural-responsive teaching. The project combines promising, but often isolated, elements from previous engineering education professional development to give teachers a) pedagogical and content knowledge, b) culturally-relevant pedagogy that is inclusive of indigenous students, c) a supportive professional learning community, d) examples of project-based engineering problems implemented in real classrooms, e) extended scaffolded practice with their own classroom engineering tasks, and f) on-going support. The program is designed for teachers in rural and tribal schools with curricular materials developed collaboratively with community input to specifically address their community's unique needs. The project research team, guided by a diverse advisory board, will collect both quantitative and qualitative data in the forms of surveys, interviews, and videotaped observations to determine if and how the project is affecting classroom engineering instruction and pedagogy, as well as the sense of competence and self-efficacy of the teacher participants. The classroom engineering tasks created through this project, especially those developed to be specifically relevant to Native American and rural student populations, will be promoted and made available to other teachers through a project website, teaching practice journals, and teacher conferences.

Exploring COVID and the Effects on U.S. Education: Evidence from a National Survey of American Households

This study aims to understand parents' perspectives on the educational impacts of COVID-19 by leveraging a nationally representative, longitudinal study, the Understanding America Study (UAS). The study will track educational experiences during the summer of 2020 and into the 2020-21 school year and analyze outcomes overall and for key demographic groups of interest.

Award Number: 
2037179
Funding Period: 
Wed, 07/15/2020 to Wed, 06/30/2021
Full Description: 

The COVID-19 epidemic has been a tremendous disruption to the education of U.S. students and their families, and early evidence suggests that this disruption has been unequally felt across households by income and race/ethnicity. While other ongoing data collection efforts focus on understanding this disruption from the perspective of students or educators, less is known about the impact of COVID-19 on children's prek-12 educational experiences as reported by their parents, especially in STEM subjects. This study aims to understand parents' perspectives on the educational impacts of COVID-19 by leveraging a nationally representative, longitudinal study, the Understanding America Study (UAS). The study will track educational experiences during the summer of 2020 and into the 2020-21 school year and analyze outcomes overall and for key demographic groups of interest.

Since March of 2020, the UAS has been tracking the educational impacts of COVID-19 for a nationally representative sample of approximately 1,500 households with preK-12 children. Early results focused on quantifying the digital divide and documenting the receipt of important educational serviceslike free meals and special education servicesafter COVID-19 began. This project will support targeted administration of UAS questions to parents about students' learning experiences and engagement, overall and in STEM subjects, data analysis, and dissemination of results to key stakeholder groups. Findings will be reported overall and across key demographic groups including ethnicity, disability, urbanicity, and socioeconomic status. The grant will also support targeted research briefs addressing pressing policy questions aimed at supporting intervention strategies in states, districts, and schools moving forward. Widespread dissemination will take place through existing networks and in collaboration with other research projects focused on understanding the COVID-19 crisis. All cross-sectional and longitudinal UAS data files will be publicly available shortly after conclusion of administration so that other researchers can explore the correlates of, and outcomes associated with, COVID-19.

Supporting Students' Language, Knowledge, and Culture through Science

This project will test and refine a teaching model that brings together current research about the role of language in science learning, the role of cultural connections in students' science engagement, and how students' science knowledge builds over time. The outcome of this project will be to provide an integrated framework that can guide current and future science teachers in preparing all students with the conceptual and linguistic practices they will need to succeed in school and in the workplace.

Lead Organization(s): 
Award Number: 
2010633
Funding Period: 
Tue, 09/01/2020 to Sat, 08/31/2024
Full Description: 

The Language, Culture, and Knowledge-building through Science project seeks to explore and positively influence the work of science teachers at the intersection of three significant and ongoing challenges affecting U.S. STEM education. First, U.S. student demographics are rapidly changing, with an increasing number of students learning STEM subjects in their second language. This change means that all teachers need new skills for meeting students where they currently are, linguistically, culturally, and in terms of prior science knowledge. Second, the needs and opportunities of the national STEM workforce are changing rapidly within a shifting employment landscape. This shift means that teachers need to better understand future job opportunities and the knowledge and skills that will be necessary in those careers. Third, academic expectations in schools have changed, driven by changes in education standards. These new expectations mean that teachers need new skills to support all students to master a range of practices that are both conceptual and linguistic. To address these challenges, teachers require new models that bring together current research about the role of language in science learning, the role of cultural connections in students' science engagement, and how students' science knowledge builds over time. This project begins with such an initial model, developed collaboratively with science teachers in a prior project. The model will be rigorously tested and refined in a new geographic and demographic context. The outcome will be to provide an integrated framework that can guide current and future science teachers in preparing all students with the conceptual and linguistic practices they will need to succeed in school and in the workplace.

This project model starts with three theoretical constructs that have been integrated into an innovative framework of nine practices. These practices guide teachers in how to simultaneously support students' language development, cultural sustenance, and knowledge building through science with a focus on supporting and challenging multilingual learners. The project uses a functional view of language development, which highlights the need to support students in understanding both how and why to make shifts in language use. For example, students' attention will be drawn to differences in language use when they shift from language that is suited to peer negotiation in a lab group to written explanations suitable for a lab report. Moving beyond a funds of knowledge approach to culture, the team view of integrating students' cultural knowledge includes strengthening the role of home knowledge in school, but also guiding students to apply school knowledge to their out-of-school interests and passions. Finally, the project team's view of cumulative knowledge building, informed by work in the sociology of knowledge, highlights the need for teachers and students to understand the norms for meaning making within a given discipline. In the case of science, the three-dimensional learning model in the Next Generation Science Standards makes these disciplinary norms visible and serves as a launching point for the project's work. Teachers will be supported to structure learning opportunities that highlight what is unique about meaning making through science. Using a range of data collection and analysis methods, the project team will study changes in teachers' practices and beliefs related to language, culture and knowledge building, as teachers work with all students, and particularly with multilingual learners. The project work will take place in both classrooms and out of class science learning settings. By working closely over several years with a group of fifty science teachers spread across the state of Oregon, the project team will develop a typology of teachers (design personas) to increase the field's understanding of how to support different teachers, given their own backgrounds, in preparing all students for the broad range of academic and occupational pathways they will encounter.

Building Environmental and Educational Technology Competence and Leadership Among Educators: An Exploration in Virtual Reality Professional Development

This project will bring locally relevant virtual reality (VR) experiences to teachers and students in areas where there is historically low participation of women and underrepresented minorities in STEM. This exploratory project will support the professional growth and development of current middle and high school STEM teachers by providing multiyear summer training and school year support around environmental sciences themed content, implementing VR in the classroom, and development of a support community for the teachers.

Lead Organization(s): 
Award Number: 
2010563
Funding Period: 
Mon, 06/15/2020 to Wed, 05/31/2023
Full Description: 

Many of the nation's most vulnerable ecosystems exist near communities with scant training opportunities for teachers and students in K-12 schools. The Louisiana wetlands is one such example. Focusing on these threatened natural environments and their connection to flooding will put science, technology, engineering, and mathematics (STEM) concepts in a real-world context that is relatable to students living in these areas while integrating virtual reality technology. This technology will allow students in rural and urban schools lacking resources for field trips to be immersed into simulated field experiences. This exploratory project will support the professional growth and development of current middle and high school STEM teachers by providing multiyear summer training and school year support around three specific areas: (1) environmental sciences themed content; (2) implementing virtual reality (VR) in the classroom, and (3) development of a support community for the teachers. Findings from this project will advance the knowledge of the most effective components in professional development for teachers to incorporate new knowledge into their classrooms. This project will bring locally relevant VR experiences to teachers and students in areas where there is historically low participation of women and underrepresented minorities in STEM. Through new partnerships formed with collaborators, the results of this project will be shared broadly in informal and formal education environments including public outreach events for an increase in public scientific literacy and public engagement.

This project will expand the understanding of the impact that a multi-layered professional development program will have on improving the self-efficacy of teachers in STEM. This project will add to the field's knowledge tied to the overall research question: What are the experiences of secondary STEM teachers in rural and urban schools who participate in a multiyear professional development (PD) program? This project will provide instructional support and PD for two cohorts of ten teachers in southeastern Louisiana. Each summer, teachers will complete a two-week blended learning PD training, and during the academic year, teachers will participate in an action research community including PD meetings and monthly Critical Friends Group meetings. A longitudinal pre-post-post design will be employed to analyze whether the proposed method improves teacher's self-efficacy, instructional practices, integration of technology, and leadership as the teachers will deploy VR training locally to grow the base of teachers integrating this technology into their curriculum. The findings of this project will improve understanding of how innovative place-based technological experiences can be brought into classrooms and shared through public engagement.

Pandemic Learning Loss in U.S. High Schools: A National Examination of Student Experiences

As a result of the COVID-19 pandemic, schools across much of the U.S. have been closed since mid-March of 2020 and many students have been attempting to continue their education away from schools. Student experiences across the country are likely to be highly variable depending on a variety of factors at the individual, home, school, district, and state levels. This project will use two, nationally representative, existing databases of high school students to study their experiences in STEM education during the COVID-19 pandemic.

Lead Organization(s): 
Award Number: 
2030436
Funding Period: 
Fri, 05/15/2020 to Fri, 04/30/2021
Full Description: 

As a result of the COVID-19 pandemic, schools across much of the U.S. have been closed since mid-March of 2020 and many students have been attempting to continue their education away from schools. Student experiences across the country are likely to be highly variable depending on a variety of factors at the individual, home, school, district, and state levels. This project will use two, nationally representative, existing databases of high school students to study their experiences in STEM education during the COVID-19 pandemic. The study intends to ascertain whether students are taking STEM courses in high school, the nature of the changes made to the courses, and their plans for the fall. The researchers will identify the electronic learning platforms in use, and other modifications made to STEM experiences in formal and informal settings. The study is particularly interested in finding patterns of inequities for students in various demographic groups underserved in STEM and who may be most likely to be affected by a hiatus in formal education.

This study will collect data using the AmeriSpeak Teen Panel of approximately 2,000 students aged 13 to 17 and the Infinite Campus Student Information System with a sample of approximately 2.5 million high school students. The data sets allow for relevant comparisons of student experiences prior to and during the COVID-19 pandemic and offer unique perspectives with nationally representative samples of U.S. high school students. New data collection will focus on formal and informal STEM learning opportunities, engagement, STEM course taking, the nature and frequency of instruction, interactions with teachers, interest in STEM, and career aspirations. Weighted data will be analyzed using descriptive statistics and within and between district analysis will be conducted to assess group differences. Estimates of between group pandemic learning loss will be provided with attention to demographic factors.

This RAPID award is made by the DRK-12 program in the Division of Research on Learning. The Discovery Research PreK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics by preK-12 students and teachers, through the research and development of new innovations and approaches. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for the projects.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

 

 

 

 

Systemic Transformation of Inquiry Learning Environments for STEM

This project will help teachers design and facilitate high-quality, real world STEM experiences for students, as teachers move from traditional approaches to organizing their teaching around interdisciplinary questions or problems. The project will work with building administrators to make the structural changes needed for interdisciplinary STEM instruction.

Award Number: 
2010530
Funding Period: 
Wed, 07/01/2020 to Sun, 06/30/2024
Full Description: 

This project will address a special challenge for schools: preparing educators to adopt an integrated approach to Science, Technology, Engineering and Mathematics (STEM). This is especially important for educators in underserved urban populations where teacher expertise and guidance are necessary for meaningful student engagement with STEM. Frameworks for helping teachers make these changes are urgently needed, especially approaches that support new perspectives for STEM teaching and learning at the school level. This project will help teachers design and facilitate high-quality, real world STEM experiences for students, as teachers move from traditional approaches to organizing their teaching around interdisciplinary questions or problems. The project will work with building administrators to make the structural changes needed for interdisciplinary STEM instruction. School-based instructional coaches will develop new strategies for guiding STEM teaching and sustaining the work long-term.

The project goals are to: (1) determine the feasibility and utility of the refined project approach, (2) determine the utility of the project's implementation for facilitating change in teacher knowledge and practices, (3) understand the utility of the project's implementation for fostering student change, and (4) understand the extent to which the refined project model supports organizational change in schools. To do this, the program will make its professional development more accessible by adding a blended learning component, expanding the school leadership program, formalizing a training program for new facilitators, and identifying novel ways of defining student outcomes for transdisciplinary learning. The mixed methods research design will involve twenty schools (elementary and intermediate) in New York City and New Haven, CT. A quasi-experimental, within-school rotation model will randomize grade-level participation at the school level to yield a sample of at least 240 teachers, 3,000 students, 40 school-based coaches, and 20 administrators. Quantitative data will primarily capture teacher and student outcomes, while the qualitative data will describe the context of the model implementation and provide a deeper understanding of the quantitative results.

Pages

Subscribe to Engineering