Engineering

SimScientists Assessments: Physical Science Links

The goal of this project is to develop and validate a middle school physical science assessment strand composed of four suites of simulation-based assessments for integrating into balanced (use of multiple measures), large-scale accountability science testing systems. It builds on the design templates, technical infrastructure, and evidence of the technical quality, feasibility, and instructional utility of the NSF-funded Calipers II project. The evaluation plan addresses both formative and summative aspects.

Lead Organization(s): 
Award Number: 
1221614
Funding Period: 
Mon, 10/01/2012 to Fri, 09/30/2016
Full Description: 

The goal of this project is to develop and validate a middle school physical science assessment strand composed of four suites of simulation-based assessments for integrating into balanced (use of multiple measures), large-scale accountability science testing systems. It builds on the design templates, technical infrastructure, and evidence of the technical quality, feasibility, and instructional utility of the NSF-funded Calipers II project. The assessment strand consists of multilevel (increased thinking levels) assessment designs grounded on evidence-centered principles that target practices and key disciplinary conceptual schemes, such as matter, motion, energy, and waves identified in the National Research Council report "A Framework for K-12 Science Education: Practices, Crosscutting Knowledge, and Core Ideas". The assessment model vertically links simulations (interactive with feedback to students, coaching, and reflection); curriculum-embedded assessments for formative use; unit benchmark assessment for interim summative purposes; and a set of "signature tasks" (short-term simulations on recurring problem types). Members of the Advisory Board and an Assessment Review Panel actively participate in the development and implementation of this effort. Heller Research Associates is the external evaluator. The evaluation plan addresses both formative and summative aspects.

The project's theory of action is based on model-based learning and evidence-centered design reflective of the notion that the construct of science is multidimensional, requiring (a) understanding how the components of a science conceptual system interact to produce behaviors of the system; and (b) the use of inquiry practices to investigate the dynamic behaviors and underlying components' interactions of the system. A total of eight research and development questions guide the scope of work. The questions focus on: (a) validity (substantive and technical quality) of the individual simulation assessments; and (b) classroom implementation (feasibility, fidelity, utility). The methodology for test construction and revision follows the testing standards of major professional organizations (i.e., American Educational Research Association, American Psychological Association, and National Council of Measurement in Education) through three development phases. Phase I (Assessment Development) focuses on the alignment, quality, and prototype testing, including leverage and modification of prior work, and design of new assessment suites and signature tasks. Phase II (Pilot and Validation Studies) deals with the testing of all assessments, research instruments, and study methods. Phase III (Cross-Validation Studies) substantiates the multilevel integration assessment model, cross-validates the assessments piloted in Phase II, and establishes a reliable argument that the assessments measure the intended content and inquiry practices suitable for use in district and state-level assessment systems.

Expected outcomes are: (1) a research-informed and field-tested physical science simulations-based assessment model with high potential for extended use in middle school grades; and (2) a policy brief that provides recommendations for integrating assessments into districts and state large-scale, multi-level, balanced science assessments.

Scale-Up of Selective STEM Specialty Schools: Efficacy Study

This study addresses the question: Does gaining admission to a selective STEM specialty school improve students' academic success on the SAT, SAT II, and Advanced Placement exams? Other portions of the investigation follow additional student outcomes, including: participation and success in STEM competitions; STEM publications; intentions for postsecondary STEM education and STEM careers; and initial postsecondary STEM education. This study seeks to inform considerations of the cost/benefit of directing resources to support such schools.

Lead Organization(s): 
Award Number: 
1221459
Funding Period: 
Sat, 09/01/2012 to Wed, 08/31/2016
Full Description: 

The desire to better empower high-ability STEM (science, technology, engineering, and mathematics) students has contributed in the last two decades to a jump in the creation of selective STEM specialty high schools. These schools devote all of their attention to a student body comprised only of the most talented students, ones who are most likely to be able to learn the most demanding STEM content, reach their STEM learning potential, and pursue postsecondary STEM study and careers. However, there are mixed views on the role that selective STEM specialty schools play in achieving their mission. While commenting on what is known regarding education in the STEM disciplines, a recent National Research Council report entitled "Successful K-12 STEM Education" notes that "there are no systematic data that show whether the highly capable students who attend those schools would have been just as likely to pursue a STEM major or related career or make significant contributions to technology or science if they had attended another type of school." To address the research gap, this impact study addresses the question: Does gaining admission to a selective STEM specialty school improve students' academic success on the SAT, SAT II (Math Level 2), and Advanced Placement exams (Calculus AB, Chemistry, and Physics B)? Other portions of the investigation follow additional student outcomes, including: participation and success in STEM competitions; STEM publications; National Merit scholarships; intentions for postsecondary STEM education and STEM careers; and initial postsecondary STEM education.

This study is based on the most rigorous possible design for the focal topic: a true experimental investigation of outcomes for students from many selective STEM specialty schools. The study is being accomplished through random assignment of equivalent treatment and control groups, and based on enough students to yield statistical power that can produce the most clear causal result possible. Researchers are recruiting all qualified students who apply for admission to 20 selective STEM specialty schools among the approximately 100 such schools currently in the United States. For the class entering in fall 2013, study schools are revising their routine selection process to one of assigning students for acceptance (treatment condition) or not (control condition) through random assignment of qualified applicants. Researchers, then, are following treatment students via their STEM schools and also intensively tracking all control students wherever they continue their secondary education. The study also investigates relative cost-effectiveness for educating high-achieving students in selective STEM schools versus educating them at other schools.

Since there is an acute and growing U.S. shortage of STEM professionals and technicians, it is imperative for the nation's education system to ensure that talented STEM students are reaching their maximum potential and pursuing postsecondary STEM degree programs and careers. As one strategy increasingly being used to address this need is to educate talented STEM students in selective STEM specialty schools, this study is informing considerations of the cost/benefit of directing resources to support such schools.

Partnerships for Early Childhood Curriculum Development: Readiness Through Integrative Science and Engineering (RISE)

The RISE project is creating curriculum resources for dual language learners (DLLs) in science, technology and engineering (STE). Participants include teachers in pre-K programs in the Boston area selected to target Hispanic and Chinese students and their families. The curriculum will be based on the Massachusetts framework, one of only a few states with pre-K standards. The evaluation will monitor both the progress of the research and development and the dissemination to the target audiences.

Lead Organization(s): 
Award Number: 
1221065
Funding Period: 
Sat, 09/01/2012 to Mon, 08/31/2015
Full Description: 

The RISE project is creating curriculum resources for dual language learners (DLLs) in science, technology and engineering (STE). Participants include teachers in pre-K programs in the Boston area selected to target Hispanic and Chinese students and their families. University partners include Tufts, Rutgers, Miami, and Northern Iowa, who will work with ABCD Head Start. An innovative feature is the incorporation of family funds of knowledge as a basis for the curriculum development. There are two research questions. 1. What are the most productive procedures for appropriate application of the full integrated RISE curriculum in Head Start classrooms serving DLL children? 2. What is the impact of the fully integrated RISE curriculum versus the comparison condition on teacher attitudes, classroom instruction, and quality, home-school relationships, and DLL children's STE knowledge and approaches to learning? In years 1 and two, 5 teachers are being supported, with 10 teachers in year 3. Participating parents are 40, 105, and 180 for years 1, 2 and 3. Professional development and mentoring is being provided for the teachers, and parent-teacher discussion groups are facilitating communication.

The research data is based on extensive classroom observations as well as interviews and surveys. For question 2, the project plans a quasi-experimental study of 10 RISE and 8 randomly selected comparison classrooms sampling 10 students in each classroom. Data will be analyzed with ANCOVA. The curriculum will be based on the Massachusetts framework, one of only a few states with pre-K standards. The evaluation will monitor both the progress of the research and development and the dissemination to the target audiences.

The curriculum materials are to be posted on the Tufts University website and a commercial publisher is being sought. Units are to be 6-12 weeks in duration, with a typical classroom engaging approximately four units. With the growing population of DLL students and the recognition that early childhood education in STE makes significant contributions to children's education, this project has the potential for national impacts.

Exploring the Efficacy of Engineering is Elementary (E4)

This project is developing evidence about the efficacy of the Engineering is Elementary curriculum under ideal conditions by studying the student and teacher-level effects of implementation. The project seeks to determine the core elements of the curriculum that support successful use. The findings from this study have broad implications for how engineering design curricular can be developed and implemented at the elementary level.

Lead Organization(s): 
Award Number: 
1220305
Funding Period: 
Sat, 09/15/2012 to Fri, 08/31/2018
Full Description: 

This project is developing evidence about the efficacy of the Engineering is Elementary curriculum under ideal conditions by studying the student and teacher-level effects of implementation. The rigorous level of evidence that is developed in this study has significant utility as a support for the kinds of elementary engineering curricula that are needed as the Next Generation Science Standards come online and emphasize engineering design. The study is a randomized control trial where the assignment of teachers will be to the EiE curricular materials or to a counterfactual condition, the use of more standard design engineering curricular materials. The project studies the impact of the use of the curriculum on student learning and on teachers' use of the curriculum in a fidelity of implementation study to determine the core elements of the curriculum that support successful use. The study examines the implementation of the curricular materials in a number of contexts to more fully understand the conditions under which they work best and to explicate what aspects of such project-based inquiry materials most support student learning.

This study uses a randomized cluster trial to examine the efficacy of the EiE curriculum across 75 schools in the treatment and 75 schools in the control group samples. Two teachers per school are included in one treatment/control condition per school. Outcome measures for students include performances on project-specific measures that have been examined for technical quality of validity and reliability. A set of additional research-based survey instruments validated for use in the EiE context are also used to collect data about students' attitudes, perceptions, interest and motivation toward science and engineering. A robust fidelity of implementation research plan is being implemented that will include teachers surveys, pre and post assessments, teacher logs, as well as student engineering journals and student work from classroom implementation. The fidelity of implementation is further studied with forty treatment and ten control teachers through classroom observations and interviews.

The findings from this study have broad implications for how engineering design curricular can be developed and implemented at the elementary level. Engineering design has not been emphasized in the elementary classroom, lagging behind instruction in science with which teachers are more familiar. The results of this study inform practitioners and policy makers about what works, for whom and under what conditions. Information about the different contexts in which the curriculum has been implemented supports the dissemination of evidence-based research and development practices to strengthen STEM learning for all students.

Educational Design and Development: Planning for a STEM Learning Research Transformation

This is a planning effort to explore future directions and innovations related to educational design in science, technology, engineering, and mathematics education in partnership with the International Society for Design and Development in Education. The planning activity will engage a core group of ISDDE principals in the articulation and examination of design processes for the Transforming STEM Learning program at NSF with a goal of developing an agenda for further discussion and research conceptualization.

Lead Organization(s): 
Award Number: 
1216850
Funding Period: 
Tue, 05/15/2012 to Tue, 04/30/2013

Crisis in K-16 STEM Education: A Regional Conference to Promote Local Solutions to a National Problem

This award is for the funding of a regional conference to study the future of STEM education, the impact of underrepresented and disadvantaged groups with regards to STEM, and STEM job growth and workforce development in a regional, as opposed to a national, context.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1212282
Funding Period: 
Wed, 02/01/2012 to Thu, 01/31/2013
Full Description: 

This award is for the funding of a regional conference to study the future of Science, Technology, Engineering, and Mathematics (STEM) education, the educational advancement of learners from underrepresented and disadvantaged groups with regards to STEM, and STEM job growth and workforce development in a regional, as opposed to a national, context. The project brings together regional K-16 stakeholders (teachers, administrators, policy makers, community college and four-year college faculty) with STEM education experts to address the major challenges and opportunities in supporting outstanding local programs that prepare students for STEM college-level study and careers, with a special emphasis on preparing under-represented populations. It is designed to bring together researchers and educators from the lower to mid-Hudson River Valley region of New York to layout the contours of current K-16 STEM practice, particularly from the point of view of what efforts are not working, why they are not working, and how to make these efforts work. The project is a partnership with Rockland Community College and CEJJES Institute. Approximately 150 participants from area schools and colleges in New York are expected to attend, representing approximately 200,000 K-12 public schools and 100,000 college students in a region where STEM-related industries are a prominent and growing influence.

The overarching goal of the Conference is to promote regional strategies that will enable this generation of learners, especially those from under-represented groups, to take their place in 21st century STEM careers. It is suggested that such a gathering of individuals and groups concerned with STEM education, as proposed in this project, would address four key questions: (1) How are knowledge and skill requirements for college-level study and careers in STEM changing the preparation needed for K-16? What changes need to be made with respect to curriculum, teachers and teaching, laboratories and access to resources, and in-school and out-of-school learning to improve the regional STEM outcomes? What strategies and practices can be adopted to support and advance STEM education throughout the region? (2) What are effective strategies for advancing the academic success of under-represented groups in STEM and how can they be successfully implemented in this region? (3) How does changing context for STEM education impact the knowledge and pedagogical skill requirements needed for being an effective K-16 STEM educator? What pedagogical strategies are best suited for teaching 21st century STEM skills? How well are teachers' professional development needs being met? What are some strategies for ensuring that the region?s teachers have access to the STEM professional development they need? (4) What are some current models of regional school/community/college partnerships for strengthening the K-16 STEM pipeline? How do these models address regional needs in ways that school districts cannot respond on their own? What solutions would be a good fit for this region? What unique ways in which Community Colleges and other share educational resources, serve as a STEM resource for students in middle school through college? What are the implications of this strategy for other regions concerned about K-16 STEM education?

Regional strategies offer a viable and scalable model for addressing K-16 STEM, especially when they reinforce the availability of services and support that would go beyond the reach of individual school districts. As a result of conference activities, the project will create and maintain a conference website with video capture of key elements of the presentations, conference proceedings and information and materials collected. The website will also be available for shared resources, scholarly papers, and the facilitating of future dialogue.

Morehouse College DR K-12 Pre-service STEM Teacher Initiative

This project recruited high school African American males to begin preparation for science, technology, engineering and mathematics teaching careers. The goal of the program was to recruit and prepare students for careers in secondary mathematics and science teaching thus increasing the number of African Americans students in STEM. The research will explore possible reasons why the program is or is not successful for recruiting and retaining students in STEM Teacher Education programs  

Lead Organization(s): 
Award Number: 
1119512
Funding Period: 
Fri, 07/15/2011 to Sat, 06/30/2018
Project Evaluator: 
Melissa K. Demetrikopoulos
Full Description: 

Morehouse College proposed a research and development project to recruit high school African American males to begin preparation for secondary school science, technology, engineering and mathematics(STEM) teaching as a career. The major goal of the program is to recruit and prepare students for careers in secondary mathematics and science teaching thus increasing the number of African Americans students in STEM. The research will explore possible reasons why the program is or is not successful for recruiting and retaining students in STEM Teacher Education programs including: (a) How do students who remain in STEM education differ from those who leave and how do these individual factors (e.g. student preparation, self-efficacy, course work outcomes, attitudes toward STEM/STEM education, connectivity to STEM/STEM education communities, learning styles, etc) enhance or inhibit interest in STEM teaching among African American males? (b) What organizational and programmatic factors (e.g. high school summer program, Saturday Academy, pre-freshman program, summer research experience, courses, enhanced mentoring, cyber-infrastructure, college admissions guidance, leadership training, instructional laboratory, program management, faculty/staff engagement and availability, Atlanta Public Schools and Morehouse College articulation and partnership) affect (enhance or inhibit) interest in STEM teaching among African American males?

This pre-service program for future secondary STEM teachers recruits promising African American male students in eleventh grade and prepares them for entry into college.  The program provides academic guidance and curriculum-specific activities for college readiness, and creates preparation for secondary science and math teaching careers.   This project is housed within the Division of Science and Mathematics at Morehouse College and engages in ongoing collaboration with the Atlanta Public School (APS) system and Fulton County School District (FCS). The APS-FCS-MC collaboration fosters access and success of underrepresented students through (a) early educational intervention practices; (b) enhanced academic preparation; and (c) explicit student recruitment. 

The program consists of six major program components: High School Summer Program; Saturday Academy I, II, and III; Pre-Freshman Summer Program; and Summer Research Experience, which begins in the summer between the student’s junior and senior years of high school and supports the student through his sophomore year of college.  To date, collaborations between education and STEM faculty as well as between Morehouse, APS, and FCS faculty have resulted in development and implementation of all six program components.   Students spent six weeks in an intensive summer program with a follow-up Saturday Academy during their senior year before formally beginning their academic careers at Morehouse College. The program integrates STEM education with teacher preparation and mentoring in order to develop secondary teachers who have mastery in both a STEM discipline as well as educational theory. 

This pre-service program for future teachers recruited promising eleventh grade African American male students from the Atlanta Public School District to participate in a four-year program that will track them into the Teacher Preparation program at Morehouse College. The research focuses on the utility and efficacy of early recruitment of African American male students to STEM teaching careers as a mechanism to increase the number of African American males in STEM teaching careers.

InterLACE: Interactive Learning and Collaboration Environment

This project designs, constructs, and field-tests a web-based, online collaborative environment for supporting the teaching and learning of inquiry-based high school physics. Based on an interactive digital workbook environment, the team is customizing the platform to include scaffolds and other supports for learning physics, fostering interaction and collaboration within the classroom, and facilitating a design-based approach to scientific experiments.

Lead Organization(s): 
Award Number: 
1119321
Funding Period: 
Thu, 09/01/2011 to Sat, 08/31/2013
Full Description: 

This project, under the Tufts University Center for Engineering Education and Outreach (CEEO) designs, constructs, and field-tests a web-based, online collaborative environment for supporting the teaching and learning of inquiry-based high school physics. Based on prior NSF-funded work on RoboBooks, an interactive digital workbook environment, the team is customizing the platform to include scaffolds and other supports for learning physics, fostering interaction and collaboration within the classroom, and facilitating a design-based approach to scientific experiments. The InterLACE team hypothesizes that technology seamlessly integrating physics content and process skills within a classroom learning activity will provide a wide variety of student benefits, ranging from improved learning outcomes and increased content knowledge to gains in attitudinal and social displays as well.

The hypothesis for this work is based on research that indicates teachers believe proper implementation of design-based, inquiry projects are time consuming and can be difficult to manage and facilitate in classrooms without great scaffolding or other supports. Using design-based research with a small number of teachers and students, the PIs iteratively develop the system and supporting materials and generate a web-based implementation that supports students through the various stages of design inquiry. A quasi-experimental trial in the final years of the project is used to determine the usability of the technology and efficacy of the system in enhancing teaching and learning. Through the tools and activities developed, the researchers anticipate showing increases in effective inquiry learning and enhanced accessibility to meet the needs of diverse learners and teachers, leading to changes in classroom practice.

Through this project the PIs (1) gain insights that will enable them to refine the InterLACE platform so it can be implemented and brought to scale in the near terms as a support for design-based inquiry science projects, and (2) advance theory, design and practice to support the design of technology-based learning environments, and (3) understand how connecting students? hypotheses, ideas, and data impacts their learning of physics content and scientific inquiry skills.

An Examination of Science and Technology Teachers' Conceptual Learning Through Concept-Based Engineering Professional Development

This project will determine the viability of an engineering concept-based approach to teacher professional development for secondary school science teachers in life science and in physical science. The project refines the conceptual base for engineering at the secondary level learning to increase the understanding of engineering concepts by the science teachers. The hypothesis is that when teachers and students engage with engineering design activities their understanding of science concepts and inquiry are also enhanced.

Project Email: 
Award Number: 
1158615
Funding Period: 
Thu, 09/01/2011 to Mon, 10/31/2011
Project Evaluator: 
Karen Peterman
Full Description: 

Technology educators from Black Hills State University and Purdue University partner with science educators from the University of Massachusetts at Boston and Stevens Institute of Technology to determine the viability of an engineering concept-based approach to teacher professional development for secondary school science teachers in life science and in physical science. The project refines the conceptual base for engineering at the secondary level learning (previously developed by the PIs) to increase the understanding of engineering concepts by the science teachers. In a pilot test of two weeks of professional development with ten teachers from each discipline, teachers become familiar with engineering concepts and study the process of infusing engineering concepts into science curricula so that they can develop modules in their discipline to be taught during the following in the school year. The following summer the teachers debrief the process and develop additional modules for their discipline. The process is revised and repeated with 22 teachers from each discipline. Teachers are explicitly provided strategies to help them meet the needs of diverse learners. The outputs of this project include: 1) a preliminary framework for secondary level engineering education to be published in both research and practitioner journals; 2) a pilot tested and validated Engineering Concept Assessment; 3) engineering-infused curriculum modules in life and physical science; and 4) a professional development model to prepare science teachers to infuse engineering in their teaching.

The project compares student learning when particular concepts in physics and biology are taught through engineering design with learning the same concepts taught an earlier group of students with present reform techniques used in the discipline. The hypothesis is that when teachers and students engage with engineering design activities their understanding of science concepts and inquiry are also enhanced. The research component of the project employs an iterative design with the design of activities followed by development and implementation. An engineering concept assessment is developed and tested to examine teacher learning and to determine how engineering concepts can be infused into the science curricula for life and physical science. Other quantitative and qualitative instruments are developed to assess the teachers? understandings of the engineering concepts and their pedagogical implications.

There is increasing emphasis on integrative STEM education. New national and international assessments are developing engineering strands and emphasizing non-routine problem solving. The framework for the Next Generation Science Standards includes engineering as one of four strands. Stand alone engineering course are not likely to be widely used. This project develops engineering infused science units and determines the professional development needed to use them effectively.

Project ATOMS: Accomplished Elementary Teachers of Mathematics and Science

The project is studying the impact of the mathematics and science intensive pre-service preparation program for elementary school teachers.  The project includes assessments of pre-service teachers' math and science content, teacher performance, self-report surveys, and teacher interviews. Each of the study dimensions (Knowledge Dimension, Teaching Performance, and Perspectives on the Program) will be assessed at three time points across this longitudinal study, providing a model for elementary teacher development of STEM teaching.

Partner Organization(s): 
Award Number: 
1118894
Funding Period: 
Thu, 09/01/2011 to Sat, 08/31/2019
Full Description: 

The project is studying the impact of the mathematics and science intensive pre-service preparation program for elementary school teachers at North Carolina State University called the Accomplished Elementary Teachers of Mathematics and Science (ATOMS). Faculty in NCSU's Department of Elementary Education, researchers at the Duke University Sanford School of Public Policy's Education Research Data Center and the NC State College Professional Education Office are involved in conducting this project.

The project includes assessments of pre-service teachers' math and science content, teacher performance, self-report surveys, and teacher interviews. Researchers are also tracking participants' perspectives on the program and comparing knowledge dimensions and teaching performance of a sub-sample of ATOMS teachers to a similar group of non-ATOMS teachers. Each of the study dimensions (Knowledge Dimension, Teaching Performance, and Perspectives on the Program) will be assessed at three time points across this longitudinal study, providing a model for elementary teacher development of STEM teaching.

The study has potential to advance current understanding regarding teacher preparation, especially in terms of supporting elementary teachers' instruction in science and math. The project is also innovative and potentially transformative by asking interesting and pertinent questions of how teachers can affect the learning of their students. Besides generating new knowledge, this project also has the potential to impact STEM education research. The ATOMS pre-service teacher preparation program may serve as a model for effective pre-service teacher education across the nation if the researchers can clearly demonstrate the effect of participating in the program in changing teachers' knowledge, attitudes, and skills, as well as their students' achievement. Investigators propose the dissemination of findings to both K-12 audiences and institutions of higher education. Additionally, key findings will be bulleted for policy makers in brief reports or brochures sent to deans of Colleges of Education nationwide, highlighting recommendations based on the findings.

Pages

Subscribe to Engineering