Engineering

Identifying and Measuring the Implementation and Impact of STEM School Models

The goal of this Transforming STEM Learning project is to comprehensively describe models of 20 inclusive STEM high schools in five states (California, New Mexico, New York, Ohio, and Texas), measure the factors that affect their implementation; and examine the relationships between these, the model components, and a range of student outcomes. The project is grounded in theoretical frameworks and research related to learning conditions and fidelity of implementation.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1238552
Funding Period: 
Mon, 10/01/2012 to Fri, 09/30/2016
Full Description: 

The goal of this Transforming STEM Learning project is to comprehensively describe models of 20 inclusive STEM high schools in five states (California, New Mexico, New York, Ohio, and Texas), measure the factors that affect their implementation; and examine the relationships between these, the model components, and a range of student outcomes. The project is grounded in theoretical frameworks and research related to learning conditions and fidelity of implementation.

The study employs a longitudinal, mixed-methods research design over four years. Research questions are: (1) What are the intended components of each inclusive STEM school model?; (2) What is the status of the intended components of each STEM school model?; (3) What are the contexts and conditions that contribute to and inhibit the implementation of components that comprise the STEM schools' models?; and (4) What components are most closely related to desired student outcomes in STEM schools? Data gathering strategies include: (a) analyses of school components (e.g., structures, interactions, practices); (b) measures of the actual implementation of components through teacher, school principals, and student questionnaires, observation protocols, teacher focus groups, and interviews; (c) identification of contextual conditions that contribute to or inhibit implementation using a framework inclusive of characteristics of the innovation, individual users, leadership, organization, and school environment using questionnaires and interviews; and (d) measuring student outcomes using four cohorts of 9-12 students, including standardized test assessment systems, grades, student questionnaires (e.g., students' perceptions of schools and teachers, self-efficacy), and postsecondary questionnaires. Quantitative data analysis strategies include: (a) assessment of validity and reliability of items measuring the implementation status of participating schools; (b) exploratory factor analysis to examine underlying dimensions of implementation and learning conditions; and (c) development of school profiles, and 2- and 3-level Hierarchical Linear Modeling to analyze relationships between implementation and type of school model. Qualitative data analysis strategies include:(a) descriptions of intra- and inter-school implementation and factor themes, (b) coding, and (c) narrative analysis.

Expected outcomes are: (a) research-informed characterizations of the range of inclusive STEM high school models emerging across the country; (b) identification of components of STEM high school models important for accomplishing a range of desired student achievement; (c) descriptions of contexts and conditions that promote or inhibit the implementation of innovative STEM teaching and learning; (d) instruments for measuring enactment of model components and the learning environments that affect them; and (e) methodological approaches for examining relationships between model components and student achievement.

Transforming STEM Competitions into Collaboratives: Developing eCrafting Collabs for Learning with Electronic Textiles

This project supports the development of technological fluency and understanding of STEM concepts through the implementation of design collaboratives that use eCrafting Collabs as the medium within which to work with middle and high school students, parents and the community. The examine how youth at ages 10-16 and families in schools, clubs, museums and community groups learn together how to create e-textile artifacts that incorporate embedded computers, sensors and actuators.

Lead Organization(s): 
Award Number: 
1238172
Funding Period: 
Mon, 10/01/2012 to Tue, 09/30/2014
Full Description: 

This project supports the development of technological fluency and understanding of STEM concepts through the implementation of design collaboratives that use eCrafting Collabs as the medium within which to work with middle and high school students, parents and the community. The researchers from the University of Pennsylvania and the Franklin Institute combine expertise in learning sciences, digital media design, computer science and informal science education to examine how youth at ages 10-16 and families in schools, clubs, museums and community groups learn together how to create e-textile artifacts that incorporate embedded computers, sensors and actuators. The project investigates the feasibility of implementing these collaboratives using eCrafting via three models of participation, individual, structured group and cross-generational community groups. They are designing a portal through which the collaborative can engage in critique and sharing of their designs as part of their efforts to build a model process by which scientific and engineered product design and analysis can be made available to multiple audiences.

The project engages participants through middle and high school elective classes and through the workshops conducted by a number of different organizations including the Franklin Institute, Techgirlz, the Hacktory and schools in Philadelphia. Participants can engage in the eCrafting Collabs through individual, collective and community design challenges that are established by the project. Participants learn about e-textile design and about circuitry and programming using either ModKit or the text-based Arduino. The designs are shared through the eCrafting Collab portal and participants are required to provide feedback and critique. Researchers are collecting data on learner identity in relation to STEM and computing, individual and collective participation in design and student understanding of circuitry and programming. The project is an example of a scalable intervention to engage students, families and communities in developing technological flexibility.

This research and development project provides a resource that engages students in middle and high schools in technology rich collaborative environments that are alternatives to other sorts of science fairs and robotic competitions. The resources developed during the project will inform how such an informal/formal blend of student engagement might be scaled to expand the experiences of populations of underserved groups, including girls. The study is conducting an examination of the new types of learning activities that are multiplying across the country with a special focus on cross-generational learning.

Community-Based Engineering Design Challenges for Adolescent English Learners

This study is based on a theoretical model that embeds engineering design within social, cultural, and linguistic activity, seeking to understand (a) how adolescent English learners draw from various linguistic, representational, and social resources as they work toward solving community-based engineering design challenges; (b) the problems they face in working on the challenges and how they seek to overcome those problems; and (c) adolescents' willingness to conceptualize themselves as future engineers.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1222566
Funding Period: 
Mon, 10/01/2012 to Wed, 09/30/2015
Full Description: 

The purpose of this exploratory study is to conduct in-depth ethnographic studies in Latino neighborhoods, documenting the funds of knowledge, social networks, and linguistic and representational repertoires that are available in the adolescents' online and offline communities. This study is based on a theoretical model that embeds engineering design within social, cultural, and linguistic activity, seeking to understand (a) how adolescent English learners draw from various linguistic, representational, and social resources as they work toward solving community-based engineering design challenges; (b) the problems they face in working on the challenges and how they seek to overcome those problems; and (c) adolescents' willingness to conceptualize themselves as future engineers before and after participating in the project.

The ethnographic research is being conducted over the course of three years. The first year includes a pilot study in which approximately four Latino adolescents from the same community identify an engineering design project and work toward implementing it. Upon completion of the pilot project, the advisory committee reviews the data collection instruments, the observation and interview techniques, and the data analysis methods. Year two includes a scaled-up version of the ethnography, in which two groups of five to seven adolescents identify a need in their respective communities and spend the remainder of the school year addressing that need through an engineering design. Student participants are members of both the MESA and Upward Bound programs which target ethnically diverse adolescents. In all, up to 18 participants (4 in the first year and 14 in the second year) are selected through a combination of direct recruitment and peer recommendations.

This study will generate frameworks for understanding the types of social resources and the types of literacy practices that are relevant to engineering processes. The knowledge generated in this study is essential for creating future curricular materials and professional development models that will enhance engineering education for culturally diverse students.

Evaluation of the Sustainability and Effectiveness of Inquiry-Based Advanced Placement Science Courses: Evidence From an In-Depth Formative Evaluation and Randomized Controlled Study

This study examines the impact of the newly revised Advanced Placement (AP) Biology and Chemistry courses on students' understanding of and ability to utilize scientific inquiry, on students' confidence in engaging in college-level material, and on students’ enrollment and persistence in college STEM majors. The project provides estimates of the impact of students' AP-course taking on their progress into postsecondary educational experiences and their intent to continue to prepare to be future engineers and scientists.

Award Number: 
1220092
Funding Period: 
Sat, 09/15/2012 to Wed, 08/31/2016
Full Description: 

This study examines the impact of the newly revised Advanced Placement (AP) Biology and Chemistry courses on students' understanding of and ability to apply scientific inquiry, on students' confidence in successfully engaging in college-level material, and on students enrollment and persistence in college STEM majors. AP Biology and Chemistry courses represent an important educational program that operates at a large scale across the country. The extent to which the AP curricula vary in implementation across the schools in the study is also examined to determine the range of students' opportunity to learn the disciplinary content and the knowledge and skills necessary to engage in inquiry in science. Schools that are newly implementing AP courses are participants in this research and the challenges and successes that they experience are also a component of the research plan. Researchers at the University of Washington, George Washington University and SRI International are conducting the study.

The research design for this study includes both formative components and a randomized control experiment. Formative elements include observations, interviews and surveys of teachers and students in the AP courses studied. The experimental design includes the random assignment of students to the AP offered and follows the performances of the treatment and control students in two cohorts into their matriculation into postsecondary educational experiences. Surveys measure students' experiences in the AP courses, their motivations to study AP science, the level of stress they experience in their high school coursework and their scientific inquiry skills and depth of disciplinary knowledge. The study examines the majors chosen by those students who enter into colleges and universities to ascertain the extent to which they continue in science and engineering.

This project informs educators about the challenges and successes schools encounter when they expand access to AP courses. The experiences of the teachers who will be teaching students with variable preparation inform future needs for professional development and support. The project provides estimates of the impact of students' AP-course taking on their progress into postsecondary educational experiences and their intent to continue to prepare to be future engineers and scientists. It informs policy efforts to improve the access to more rigorous advanced courses in STEM and provides strong experimental evidence of the impact of AP course taking. The project has the potential to demonstrate to educational researchers how to study an educational program that operates at scale.

SimScientists Assessments: Physical Science Links

The goal of this project is to develop and validate a middle school physical science assessment strand composed of four suites of simulation-based assessments for integrating into balanced (use of multiple measures), large-scale accountability science testing systems. It builds on the design templates, technical infrastructure, and evidence of the technical quality, feasibility, and instructional utility of the NSF-funded Calipers II project. The evaluation plan addresses both formative and summative aspects.

Lead Organization(s): 
Award Number: 
1221614
Funding Period: 
Mon, 10/01/2012 to Fri, 09/30/2016
Full Description: 

The goal of this project is to develop and validate a middle school physical science assessment strand composed of four suites of simulation-based assessments for integrating into balanced (use of multiple measures), large-scale accountability science testing systems. It builds on the design templates, technical infrastructure, and evidence of the technical quality, feasibility, and instructional utility of the NSF-funded Calipers II project. The assessment strand consists of multilevel (increased thinking levels) assessment designs grounded on evidence-centered principles that target practices and key disciplinary conceptual schemes, such as matter, motion, energy, and waves identified in the National Research Council report "A Framework for K-12 Science Education: Practices, Crosscutting Knowledge, and Core Ideas". The assessment model vertically links simulations (interactive with feedback to students, coaching, and reflection); curriculum-embedded assessments for formative use; unit benchmark assessment for interim summative purposes; and a set of "signature tasks" (short-term simulations on recurring problem types). Members of the Advisory Board and an Assessment Review Panel actively participate in the development and implementation of this effort. Heller Research Associates is the external evaluator. The evaluation plan addresses both formative and summative aspects.

The project's theory of action is based on model-based learning and evidence-centered design reflective of the notion that the construct of science is multidimensional, requiring (a) understanding how the components of a science conceptual system interact to produce behaviors of the system; and (b) the use of inquiry practices to investigate the dynamic behaviors and underlying components' interactions of the system. A total of eight research and development questions guide the scope of work. The questions focus on: (a) validity (substantive and technical quality) of the individual simulation assessments; and (b) classroom implementation (feasibility, fidelity, utility). The methodology for test construction and revision follows the testing standards of major professional organizations (i.e., American Educational Research Association, American Psychological Association, and National Council of Measurement in Education) through three development phases. Phase I (Assessment Development) focuses on the alignment, quality, and prototype testing, including leverage and modification of prior work, and design of new assessment suites and signature tasks. Phase II (Pilot and Validation Studies) deals with the testing of all assessments, research instruments, and study methods. Phase III (Cross-Validation Studies) substantiates the multilevel integration assessment model, cross-validates the assessments piloted in Phase II, and establishes a reliable argument that the assessments measure the intended content and inquiry practices suitable for use in district and state-level assessment systems.

Expected outcomes are: (1) a research-informed and field-tested physical science simulations-based assessment model with high potential for extended use in middle school grades; and (2) a policy brief that provides recommendations for integrating assessments into districts and state large-scale, multi-level, balanced science assessments.

Scale-Up of Selective STEM Specialty Schools: Efficacy Study

This study addresses the question: Does gaining admission to a selective STEM specialty school improve students' academic success on the SAT, SAT II, and Advanced Placement exams? Other portions of the investigation follow additional student outcomes, including: participation and success in STEM competitions; STEM publications; intentions for postsecondary STEM education and STEM careers; and initial postsecondary STEM education. This study seeks to inform considerations of the cost/benefit of directing resources to support such schools.

Lead Organization(s): 
Award Number: 
1221459
Funding Period: 
Sat, 09/01/2012 to Wed, 08/31/2016
Full Description: 

The desire to better empower high-ability STEM (science, technology, engineering, and mathematics) students has contributed in the last two decades to a jump in the creation of selective STEM specialty high schools. These schools devote all of their attention to a student body comprised only of the most talented students, ones who are most likely to be able to learn the most demanding STEM content, reach their STEM learning potential, and pursue postsecondary STEM study and careers. However, there are mixed views on the role that selective STEM specialty schools play in achieving their mission. While commenting on what is known regarding education in the STEM disciplines, a recent National Research Council report entitled "Successful K-12 STEM Education" notes that "there are no systematic data that show whether the highly capable students who attend those schools would have been just as likely to pursue a STEM major or related career or make significant contributions to technology or science if they had attended another type of school." To address the research gap, this impact study addresses the question: Does gaining admission to a selective STEM specialty school improve students' academic success on the SAT, SAT II (Math Level 2), and Advanced Placement exams (Calculus AB, Chemistry, and Physics B)? Other portions of the investigation follow additional student outcomes, including: participation and success in STEM competitions; STEM publications; National Merit scholarships; intentions for postsecondary STEM education and STEM careers; and initial postsecondary STEM education.

This study is based on the most rigorous possible design for the focal topic: a true experimental investigation of outcomes for students from many selective STEM specialty schools. The study is being accomplished through random assignment of equivalent treatment and control groups, and based on enough students to yield statistical power that can produce the most clear causal result possible. Researchers are recruiting all qualified students who apply for admission to 20 selective STEM specialty schools among the approximately 100 such schools currently in the United States. For the class entering in fall 2013, study schools are revising their routine selection process to one of assigning students for acceptance (treatment condition) or not (control condition) through random assignment of qualified applicants. Researchers, then, are following treatment students via their STEM schools and also intensively tracking all control students wherever they continue their secondary education. The study also investigates relative cost-effectiveness for educating high-achieving students in selective STEM schools versus educating them at other schools.

Since there is an acute and growing U.S. shortage of STEM professionals and technicians, it is imperative for the nation's education system to ensure that talented STEM students are reaching their maximum potential and pursuing postsecondary STEM degree programs and careers. As one strategy increasingly being used to address this need is to educate talented STEM students in selective STEM specialty schools, this study is informing considerations of the cost/benefit of directing resources to support such schools.

Partnerships for Early Childhood Curriculum Development: Readiness Through Integrative Science and Engineering (RISE)

The RISE project is creating curriculum resources for dual language learners (DLLs) in science, technology and engineering (STE). Participants include teachers in pre-K programs in the Boston area selected to target Hispanic and Chinese students and their families. The curriculum will be based on the Massachusetts framework, one of only a few states with pre-K standards. The evaluation will monitor both the progress of the research and development and the dissemination to the target audiences.

Lead Organization(s): 
Award Number: 
1221065
Funding Period: 
Sat, 09/01/2012 to Mon, 08/31/2015
Full Description: 

The RISE project is creating curriculum resources for dual language learners (DLLs) in science, technology and engineering (STE). Participants include teachers in pre-K programs in the Boston area selected to target Hispanic and Chinese students and their families. University partners include Tufts, Rutgers, Miami, and Northern Iowa, who will work with ABCD Head Start. An innovative feature is the incorporation of family funds of knowledge as a basis for the curriculum development. There are two research questions. 1. What are the most productive procedures for appropriate application of the full integrated RISE curriculum in Head Start classrooms serving DLL children? 2. What is the impact of the fully integrated RISE curriculum versus the comparison condition on teacher attitudes, classroom instruction, and quality, home-school relationships, and DLL children's STE knowledge and approaches to learning? In years 1 and two, 5 teachers are being supported, with 10 teachers in year 3. Participating parents are 40, 105, and 180 for years 1, 2 and 3. Professional development and mentoring is being provided for the teachers, and parent-teacher discussion groups are facilitating communication.

The research data is based on extensive classroom observations as well as interviews and surveys. For question 2, the project plans a quasi-experimental study of 10 RISE and 8 randomly selected comparison classrooms sampling 10 students in each classroom. Data will be analyzed with ANCOVA. The curriculum will be based on the Massachusetts framework, one of only a few states with pre-K standards. The evaluation will monitor both the progress of the research and development and the dissemination to the target audiences.

The curriculum materials are to be posted on the Tufts University website and a commercial publisher is being sought. Units are to be 6-12 weeks in duration, with a typical classroom engaging approximately four units. With the growing population of DLL students and the recognition that early childhood education in STE makes significant contributions to children's education, this project has the potential for national impacts.

Exploring the Efficacy of Engineering is Elementary (E4)

This project is developing evidence about the efficacy of the Engineering is Elementary curriculum under ideal conditions by studying the student and teacher-level effects of implementation. The project seeks to determine the core elements of the curriculum that support successful use. The findings from this study have broad implications for how engineering design curricular can be developed and implemented at the elementary level.

Lead Organization(s): 
Award Number: 
1220305
Funding Period: 
Sat, 09/15/2012 to Fri, 08/31/2018
Full Description: 

This project is developing evidence about the efficacy of the Engineering is Elementary curriculum under ideal conditions by studying the student and teacher-level effects of implementation. The rigorous level of evidence that is developed in this study has significant utility as a support for the kinds of elementary engineering curricula that are needed as the Next Generation Science Standards come online and emphasize engineering design. The study is a randomized control trial where the assignment of teachers will be to the EiE curricular materials or to a counterfactual condition, the use of more standard design engineering curricular materials. The project studies the impact of the use of the curriculum on student learning and on teachers' use of the curriculum in a fidelity of implementation study to determine the core elements of the curriculum that support successful use. The study examines the implementation of the curricular materials in a number of contexts to more fully understand the conditions under which they work best and to explicate what aspects of such project-based inquiry materials most support student learning.

This study uses a randomized cluster trial to examine the efficacy of the EiE curriculum across 75 schools in the treatment and 75 schools in the control group samples. Two teachers per school are included in one treatment/control condition per school. Outcome measures for students include performances on project-specific measures that have been examined for technical quality of validity and reliability. A set of additional research-based survey instruments validated for use in the EiE context are also used to collect data about students' attitudes, perceptions, interest and motivation toward science and engineering. A robust fidelity of implementation research plan is being implemented that will include teachers surveys, pre and post assessments, teacher logs, as well as student engineering journals and student work from classroom implementation. The fidelity of implementation is further studied with forty treatment and ten control teachers through classroom observations and interviews.

The findings from this study have broad implications for how engineering design curricular can be developed and implemented at the elementary level. Engineering design has not been emphasized in the elementary classroom, lagging behind instruction in science with which teachers are more familiar. The results of this study inform practitioners and policy makers about what works, for whom and under what conditions. Information about the different contexts in which the curriculum has been implemented supports the dissemination of evidence-based research and development practices to strengthen STEM learning for all students.

Educational Design and Development: Planning for a STEM Learning Research Transformation

This is a planning effort to explore future directions and innovations related to educational design in science, technology, engineering, and mathematics education in partnership with the International Society for Design and Development in Education. The planning activity will engage a core group of ISDDE principals in the articulation and examination of design processes for the Transforming STEM Learning program at NSF with a goal of developing an agenda for further discussion and research conceptualization.

Lead Organization(s): 
Award Number: 
1216850
Funding Period: 
Tue, 05/15/2012 to Tue, 04/30/2013

Crisis in K-16 STEM Education: A Regional Conference to Promote Local Solutions to a National Problem

This award is for the funding of a regional conference to study the future of STEM education, the impact of underrepresented and disadvantaged groups with regards to STEM, and STEM job growth and workforce development in a regional, as opposed to a national, context.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1212282
Funding Period: 
Wed, 02/01/2012 to Thu, 01/31/2013
Full Description: 

This award is for the funding of a regional conference to study the future of Science, Technology, Engineering, and Mathematics (STEM) education, the educational advancement of learners from underrepresented and disadvantaged groups with regards to STEM, and STEM job growth and workforce development in a regional, as opposed to a national, context. The project brings together regional K-16 stakeholders (teachers, administrators, policy makers, community college and four-year college faculty) with STEM education experts to address the major challenges and opportunities in supporting outstanding local programs that prepare students for STEM college-level study and careers, with a special emphasis on preparing under-represented populations. It is designed to bring together researchers and educators from the lower to mid-Hudson River Valley region of New York to layout the contours of current K-16 STEM practice, particularly from the point of view of what efforts are not working, why they are not working, and how to make these efforts work. The project is a partnership with Rockland Community College and CEJJES Institute. Approximately 150 participants from area schools and colleges in New York are expected to attend, representing approximately 200,000 K-12 public schools and 100,000 college students in a region where STEM-related industries are a prominent and growing influence.

The overarching goal of the Conference is to promote regional strategies that will enable this generation of learners, especially those from under-represented groups, to take their place in 21st century STEM careers. It is suggested that such a gathering of individuals and groups concerned with STEM education, as proposed in this project, would address four key questions: (1) How are knowledge and skill requirements for college-level study and careers in STEM changing the preparation needed for K-16? What changes need to be made with respect to curriculum, teachers and teaching, laboratories and access to resources, and in-school and out-of-school learning to improve the regional STEM outcomes? What strategies and practices can be adopted to support and advance STEM education throughout the region? (2) What are effective strategies for advancing the academic success of under-represented groups in STEM and how can they be successfully implemented in this region? (3) How does changing context for STEM education impact the knowledge and pedagogical skill requirements needed for being an effective K-16 STEM educator? What pedagogical strategies are best suited for teaching 21st century STEM skills? How well are teachers' professional development needs being met? What are some strategies for ensuring that the region?s teachers have access to the STEM professional development they need? (4) What are some current models of regional school/community/college partnerships for strengthening the K-16 STEM pipeline? How do these models address regional needs in ways that school districts cannot respond on their own? What solutions would be a good fit for this region? What unique ways in which Community Colleges and other share educational resources, serve as a STEM resource for students in middle school through college? What are the implications of this strategy for other regions concerned about K-16 STEM education?

Regional strategies offer a viable and scalable model for addressing K-16 STEM, especially when they reinforce the availability of services and support that would go beyond the reach of individual school districts. As a result of conference activities, the project will create and maintain a conference website with video capture of key elements of the presentations, conference proceedings and information and materials collected. The website will also be available for shared resources, scholarly papers, and the facilitating of future dialogue.

Pages

Subscribe to Engineering