Quantitative

An Examination of Science and Technology Teachers' Conceptual Learning Through Concept-Based Engineering Professional Development

This project will determine the viability of an engineering concept-based approach to teacher professional development for secondary school science teachers in life science and in physical science. The project refines the conceptual base for engineering at the secondary level learning to increase the understanding of engineering concepts by the science teachers. The hypothesis is that when teachers and students engage with engineering design activities their understanding of science concepts and inquiry are also enhanced.

Project Email: 
Award Number: 
1158615
Funding Period: 
Thu, 09/01/2011 to Mon, 10/31/2011
Project Evaluator: 
Karen Peterman
Full Description: 

Technology educators from Black Hills State University and Purdue University partner with science educators from the University of Massachusetts at Boston and Stevens Institute of Technology to determine the viability of an engineering concept-based approach to teacher professional development for secondary school science teachers in life science and in physical science. The project refines the conceptual base for engineering at the secondary level learning (previously developed by the PIs) to increase the understanding of engineering concepts by the science teachers. In a pilot test of two weeks of professional development with ten teachers from each discipline, teachers become familiar with engineering concepts and study the process of infusing engineering concepts into science curricula so that they can develop modules in their discipline to be taught during the following in the school year. The following summer the teachers debrief the process and develop additional modules for their discipline. The process is revised and repeated with 22 teachers from each discipline. Teachers are explicitly provided strategies to help them meet the needs of diverse learners. The outputs of this project include: 1) a preliminary framework for secondary level engineering education to be published in both research and practitioner journals; 2) a pilot tested and validated Engineering Concept Assessment; 3) engineering-infused curriculum modules in life and physical science; and 4) a professional development model to prepare science teachers to infuse engineering in their teaching.

The project compares student learning when particular concepts in physics and biology are taught through engineering design with learning the same concepts taught an earlier group of students with present reform techniques used in the discipline. The hypothesis is that when teachers and students engage with engineering design activities their understanding of science concepts and inquiry are also enhanced. The research component of the project employs an iterative design with the design of activities followed by development and implementation. An engineering concept assessment is developed and tested to examine teacher learning and to determine how engineering concepts can be infused into the science curricula for life and physical science. Other quantitative and qualitative instruments are developed to assess the teachers? understandings of the engineering concepts and their pedagogical implications.

There is increasing emphasis on integrative STEM education. New national and international assessments are developing engineering strands and emphasizing non-routine problem solving. The framework for the Next Generation Science Standards includes engineering as one of four strands. Stand alone engineering course are not likely to be widely used. This project develops engineering infused science units and determines the professional development needed to use them effectively.

Project ATOMS: Accomplished Elementary Teachers of Mathematics and Science

The project is studying the impact of the mathematics and science intensive pre-service preparation program for elementary school teachers.  The project includes assessments of pre-service teachers' math and science content, teacher performance, self-report surveys, and teacher interviews. Each of the study dimensions (Knowledge Dimension, Teaching Performance, and Perspectives on the Program) will be assessed at three time points across this longitudinal study, providing a model for elementary teacher development of STEM teaching.

Partner Organization(s): 
Award Number: 
1118894
Funding Period: 
Thu, 09/01/2011 to Sat, 08/31/2019
Full Description: 

The project is studying the impact of the mathematics and science intensive pre-service preparation program for elementary school teachers at North Carolina State University called the Accomplished Elementary Teachers of Mathematics and Science (ATOMS). Faculty in NCSU's Department of Elementary Education, researchers at the Duke University Sanford School of Public Policy's Education Research Data Center and the NC State College Professional Education Office are involved in conducting this project.

The project includes assessments of pre-service teachers' math and science content, teacher performance, self-report surveys, and teacher interviews. Researchers are also tracking participants' perspectives on the program and comparing knowledge dimensions and teaching performance of a sub-sample of ATOMS teachers to a similar group of non-ATOMS teachers. Each of the study dimensions (Knowledge Dimension, Teaching Performance, and Perspectives on the Program) will be assessed at three time points across this longitudinal study, providing a model for elementary teacher development of STEM teaching.

The study has potential to advance current understanding regarding teacher preparation, especially in terms of supporting elementary teachers' instruction in science and math. The project is also innovative and potentially transformative by asking interesting and pertinent questions of how teachers can affect the learning of their students. Besides generating new knowledge, this project also has the potential to impact STEM education research. The ATOMS pre-service teacher preparation program may serve as a model for effective pre-service teacher education across the nation if the researchers can clearly demonstrate the effect of participating in the program in changing teachers' knowledge, attitudes, and skills, as well as their students' achievement. Investigators propose the dissemination of findings to both K-12 audiences and institutions of higher education. Additionally, key findings will be bulleted for policy makers in brief reports or brochures sent to deans of Colleges of Education nationwide, highlighting recommendations based on the findings.

An Innovative Approach to Earth Science Teacher Preparation: Uniting Science, Informal Science Education, and Schools to Raise Student Achievement

The American Museum of Natural History in New York City, in partnership with New York University, and in collaboration with five high-needs schools, is developing, implementing, and researching a five-year pilot Master of Arts in Teaching (MAT) program in Earth Science. The program is delivered by the Museum's scientific and education teams and its evaluation covers aspects of the program from recruitment to first year of teaching.

Project Email: 
Award Number: 
1119444
Funding Period: 
Thu, 09/01/2011 to Thu, 08/31/2017
Project Evaluator: 
David Silvernail, Center for Education and Policy, University of Southern Maine
Full Description: 

The American Museum of Natural History (AMNH), in collaboration with New York University's Institute for Education and Social Policy and the University of Southern Maine Center for Evaluation and Policy, will develop and evaluate a new teacher education program model to prepare science teachers through a partnership between a world class science museum and high need schools in metropolitan New York City (NYC). This innovative pilot residency model was approved by the New York State (NYS) Board of Regents as part of the state’s Race To The Top award. The program will prepare a total of 50 candidates in two cohorts (2012 and 2013) to earn a Board of Regents-awarded Masters of Arts in Teaching (MAT) degree with a specialization in Earth Science for grades 7-12. The program focuses on Earth Science both because it is one of the greatest areas of science teacher shortages in urban areas and because AMNH has the ability to leverage the required scientific and educational resources in Earth Science and allied disciplines, including paleontology and astrophysics.

The proposed 15-month, 36-credit residency program is followed by two additional years of mentoring for new teachers. In addition to a full academic year of residency in high-needs public schools, teacher candidates will undertake two AMNH-based clinical summer residencies; a Museum Teaching Residency prior to entering their host schools, and a Museum Science Residency prior to entering the teaching profession. All courses will be taught by teams of doctoral-level educators and scientists.

The project’s research and evaluation components will examine the factors and outcomes of a program offered through a science museum working with the formal teacher preparation system in high need schools. Formative and summative evaluations will document all aspects of the program. In light of the NYS requirement that the pilot program be implemented in high-need, low-performing schools, this project has the potential to engage, motivate and improve the Earth Science achievement and interest in STEM careers of thousands of students from traditionally underrepresented populations including English language learners, special education students, and racial minority groups. In addition, this project will gather meaningful data on the role science museums can play in preparing well-qualified Earth Science teachers. The research component will examine the impact of this new teacher preparation model on student achievement in metropolitan NYC schools. More specifically, this project asks, "How do Earth Science students taught by first year AMNH MAT Earth Science teachers perform academically in comparison with students taught by first year Earth Science teachers not prepared in the AMNH program?.”

Energy: A Multidisciplinary Approach for Teachers (EMAT) Designing and Studying a Multidisciplinary, Online Course for High School Teachers

This project will iteratively design, develop, field test, refine, and rigorously study a six-unit, facilitated, online professional development (PD) course focusing on energy-related concepts in the context of alternative energy. The primary audience is high school science teachers teaching out of their field of endorsement and serving students underrepresented in the sciences. The project will investigate whether the PD will precipitate changes in teacher knowledge and practice that result in higher student achievement.

Award Number: 
1118643
Funding Period: 
Thu, 09/01/2011 to Thu, 08/31/2017
Project Evaluator: 
RMC Research Corporation
Full Description: 

The Energy: A Multidisciplinary Approach for Teachers (EMAT) project will iteratively design, develop, field test, refine, and rigorously study a seven-unit, facilitated, online professional development (PD) course focusing on energy-related concepts in the context of alternative energy. The primary audience is high school science teachers teaching out of their field of endorsement and serving students underrepresented in the sciences. The project will investigate whether the PD will precipitate changes in teacher knowledge and practice that result in higher student achievement. As a result, EMAT will improve the science achievement of underrepresented students and enhance their future participation in science. Biological Sciences Curriculum Study and partners Oregon Public Broadcasting, the National Teacher Enhancement Network, the National Renewable Energy Laboratory, the Great Lakes Bioenergy Research Center, and RMC Research Corporation bring significant resources and are highly qualified to develop and research EMAT.

The EMAT project advances knowledge in the field of teacher professional development by merging two facets of PD that have hitherto been studied separately and testing hypotheses about the degree to which this pairing enhances learning and practice. These facets are structured constructivist experiences and experiences grounded in situated cognition learning theory. Teachers reflect on research-based teaching practices in the lesson analysis process through Science Content Storyline and Student Thinking lenses. EMAT tests longitudinal impacts on teachers' content knowledge, pedagogical content knowledge, and teaching practices and students' content knowledge, contributing much needed data for future PD projects. EMAT also studies which aspects of online environments are most effective for teachers. Data collected will inform full revisions of the course and will help address significant gaps in our understanding of online PD.

EMAT advances the field's understanding of which elements of online PD are effective and the extent to which high-quality online PD translates to improved student learning. Simultaneously, the project develops and tests a scalable, flexible resource to enhance teacher learning and practice. As a result, EMAT will have a broad impact by promoting research-based teaching and learning while advancing discovery and understanding. Furthermore, by targeting the recruitment of teacher participants from large urban districts with high numbers of teachers teaching out of field, EMAT impacts students traditionally underrepresented in the sciences. EMAT will not only contribute to the research on PD, but also will be available for use in diverse settings. A facilitation guide allows the course to be freely used by school districts and teacher education and certification programs across the country. In addition, the facilitated course will be offered for graduate credit through the National Teacher Enhancement Network and will be freely available to individuals for independent study. Results of all research and evaluation will be published in science education journals and practitioner journals for teachers, and presented to PD groups at conferences. EMAT will benefit society by impacting teacher and student understanding of energy-related concepts, thereby increasing the capacity of U.S. citizens to creatively address energy challenges from a foundation of scientifically sound knowledge.

School Structure and Science Success: Organization and Leadership Influences on Student Achievement (Collaborative Research: Butler)

This project will document factors explaining variations in science achievement across schools enrolling ethnically and linguistically diverse students. The research question is: what leadership and organizational features at the school level are associated with mitigating science achievement gaps? Researchers, in collaboration with school districts, will identify school leadership practices that can be connected with reductions in achievement gaps related to student ethnicity, English fluency, and social status.

Partner Organization(s): 
Award Number: 
1338512
Funding Period: 
Fri, 02/15/2013 to Sat, 06/30/2018
Full Description: 

The School Organization and Science Achievement (SOSA) Project will document factors explaining variations in science achievement across schools enrolling ethnically and linguistically diverse students. The research question is: what leadership and organizational features at the school level are associated with mitigating science achievement gaps? Previous school effectiveness studies demonstrate school leadership and social capital influencing student achievement; the SOSA project is unique with its focus on science achievement. Researchers at the University of Connecticut and the University of South Florida St. Petersburg, in collaboration with school districts in their respective states, will identify school leadership practices that can be connected with reductions in achievement gaps related to student ethnicity, English fluency, and social status. At the conclusion of the five-year project, the findings will take the form of recommendations about leadership practices and school organization that can be implemented in other school settings.

The project uses a mixed methods design by combining statistical modeling and qualitative data. Multiple regression analyses highlight those schools populated by fifth graders that have greater or lesser achievement gaps in science. Using social capital theory (i.e., school norms, communication channels, and trustworthiness) comparisons of positive and negative outlier schools will be made via interviews of building principals, classroom teachers and community representatives. The expectation is that schools providing more equitable science experiences to all students will exhibit stronger social capital compared to buildings with disparities in science test scores across demographic categories. These insights will be supplemented by multilevel structural equation modeling to determine the strength of association between various school climate measures (e.g., teacher-to-principal trust, correspondence between teacher and principal perceptions of leadership, and school/community ties) and science achievement as measured by statewide fifth grade science tests. In addition, growth analyses will be used to detect shifts over time and provide insights about the links between policy changes or leadership adjustments, inasmuch as science achievement gaps are affected.

By working with 150 schools in two states, this collaborative research project is designed to generate findings applicable in other school systems. Particularly in settings where science achievement gaps are large, and especially when such gaps vary between schools even when the student populations are similar, the findings from this study will have practical leadership implications. Expertise in this project includes science education, educational leadership, and statistical modeling. This complementary combination increases the depth of the project's efforts along with expanding its potential impacts. Key questions addressed by this project include: to what extent is leadership in science similar to or different from leadership in other subject areas? how do variations in leadership design (e.g., top-down versus distributed leadership) contribute to reductions in science achievement gaps? to what degree can effective leadership mitigate other factors that exacerbate the challenges of providing high quality science learning experiences for every child? Findings will be disseminated via the SOSA Project website, along with leadership development strategies. Deliverables include templates to replicate the study, case studies for professional development, and strategies for supporting the development of science teacher-leaders.

This project was previously funded under award # 1119359.

Levels of Conceptual Understanding in Statistics (LOCUS)

LOCUS (Levels of Conceptual Understanding in Statistics) is an NSF Funded DRK12 project (NSF#118618) focused on developing assessments of statistical understanding. These assessments will measure students’ understanding across levels of development as identified in the Guidelines for Assessment and Instruction in Statistics Education (GAISE). The intent of these assessments is to provide teachers and researchers with a valid and reliable assessment of conceptual understanding in statistics consistent with the Common Core State Standards (CCSS).

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1118168
Funding Period: 
Thu, 09/01/2011 to Fri, 08/31/2012
Project Evaluator: 
TERC, Jim Hammerman
Full Description: 

The goal of this project is to develop two tests (instruments) to assess conceptual understanding of statistics. The instruments are based on the levels A/B and on level C of statistical understanding development as described in the American Statistical Association Guidelines for Assessment and Instruction of Statistics Education (GAISE) framework. These instruments will be used to assess knowledge of statistics by grades 6-12 students. The instruments will have multiple-choice and constructed response (CR) items. The CR items will have scoring rubrics. The assessments will be pilot tested in school districts in six states. The instruments will be used by teachers to analyze students' growth in understanding of statistics and will be useable for both formative and summative purposes. An assessment blueprint will be developed based on the GAISE framework for selecting and constructing both fixed-choice and open-ended items. An evidenced-based designed process will be used to develop the assessments. The blueprint will be used by the test development committee to develop items. These items will be reviewed by the advisory board considering the main statistics topics to be included on the assessments. Through a layering process, the assessments will be piloted, revised, and field tested with students in grades 6-12 in six states. A three-parameter IRT model will be used in analyzing the items. The work will be done by researchers at the University of Florida with the support of those at the University of Minnesota, the Educational Testing Service, and Kenyon College. Researchers from TERC will conduct a process evaluation with several feedback and redesign cycles.

The assessments will be aligned with the Common Core State Standards for mathematics (CCSSM) and made available as open-source to teachers through a website. The research team will interact with the state consortia developing assessments to measure students' attainment of the CCSSM. As such, the assessments have the potential of being used by a large proportion of students in the country. The more conceptually-based items will provide teachers with concrete examples of what statistics students in grades 6-12 should know.

Continuous Learning and Automated Scoring in Science (CLASS)

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items into the "Web-based Inquiry Science Environment" (WISE) program.

Award Number: 
1119670
Funding Period: 
Thu, 09/01/2011 to Mon, 08/31/2015
Full Description: 

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items (i.e., short essays, science narratives, concept mapping, graphing problems, and virtual experiments) into the "Web-based Inquiry Science Environment" (WISE) program. WISE is an online science-inquiry curricula that supports deep understanding through visualization of processes not directly observable, virtual experiments, graphing results, collaboration, and response to prompts for explanations. In partnership with Educational Testing Services (ETS), project goals are: (1) to develop five automated inquiry assessment activities that capture students' abilities to integrate their ideas and form coherent scientific arguments; (2) to customize WISE by incorporating automated scores; (3) to investigate how students' systematic feedback based on these scores improve their learning outcomes; and (4) to design professional development resources to help teachers use scores to improve classroom instruction, and administrators to make better informed decisions about teacher professional development and inquiry instruction. The project targets general science (life, physical, and earth) in three northern California school districts, five middle schools serving over 4,000 6th-8th grade students with diverse cultural and linguistic backgrounds, and 29 science teachers. It contributes to increase opportunities for students to improve their science achievement, and for teachers and administrators to make efficient, evidence-based decisions about high-quality teaching and learning.

A key research question guides this effort: How automated scoring of inquiry assessments can increase success for diverse students, improve teachers' instructional practices, and inform administrators' decisions about professional development, inquiry instruction, and assessment? To develop science inquiry assessment activities, scoring written responses include semantic, syntax, and structure of meaning analyses, as well as calibration of human-scored items with a computer-scoring system through the c-rater--an ETS-developed cyber learning technology. Validity studies are conducted to compare automated scores with human-scored items, teacher, district, and state scores, including sensitivity to the diverse student population. To customize the WISE curriculum, the project modifies 12 existing units and develops nine new modules. To design adaptive feedback to students, comparative studies explore options for adaptive guidance and test alternatives based on automated scores employing linear models to compare student performance across randomly assigned guidance conditions; controlling for covariates, such as prior science scores, gender, and language; and grouping comparison studies. To design teacher professional development, synthesis reports on auto-scored data are created to enable them to use evidence to guide curricular decisions, and comments' analysis to improve feedback quality. Workshops, classroom observations, and interviews are conducted to measure longitudinal teachers' change over time. To empower administrators' decision making, special data reports, using-evidence activities, individual interviews, and observation of administrators' meetings are conducted. An advisory board charged with project evaluation addresses both formative and summative aspects.

A research-informed model to improve science teaching and learning at the middle school level through cyber-enabled assessment is the main outcome of this effort. A total of 21 new, one- to three-week duration standards-based science units, each with four or more automatically scored items, serve as prototypes to improve students' performance, teachers' instructional approaches, and administrators' school policies and practices.

CAREER: Engaging Elementary Students in Data Analysis Through Study of Physical Activities

This project is investigating the learning that can take place when elementary school students are directly involved in the collection, sense-making, and analysis of real, personally-meaningful data sets. The hypotheses of this work are that by organizing elementary statistics instruction around the study of physical activities, students will have greater personal engagement in data analysis processes and that students will also develop more robust understandings of statistical ideas.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1054280
Funding Period: 
Fri, 07/01/2011 to Sat, 06/30/2018
Full Description: 

This CAREER awardee at Utah State University is investigating the learning that can take place when elementary school students are directly involved in the collection, sense-making, and analysis of real, personally-meaningful data sets. The project responds to increasing attention to data collection and analysis in elementary grades and aims to make important contributions to the knowledge base on effective approaches to these topics. The hypotheses of this work are that by organizing elementary statistics instruction around the study of physical activities, students will have greater personal engagement in data analysis processes and that students will also develop more robust understandings of statistical ideas. Students and teachers from fifth grade classrooms from several elementary schools from northern Utah, are participating in the project. This work is co-funded by the EPSCoR program.

Statistics topics include measures of center and variation. Students use pedometers, heart rate monitors, other probeware, and the TinkerPlots software. The research team investigates the influence of personal ownership and relationships to data on students' understanding of learning of elementary statistics concepts and their ability to analyze data. The research involves multi-year clinical interviews and video-recorded classroom design experiments.

Research results are expected to be published in appropriate journals and are expected to be presented at professional meetings. Lesson plans and student instructional materials related to physical activity, measures of center, and data distributions are made available for use in partner elementary schools.

Enhancing Games with Assessment and Metacognitive Emphases (EGAME)

This development and research project designs, develops, and tests a digital game-based learning environment for supporting, assessing and analyzing middle school students' conceptual knowledge in learning physics, specifically Newtonian mechanics. This research integrates work from prior findings to develop a new methodology to engage students in deep learning while diagnosing and scaffolding the learning of Newtonian mechanics.

Lead Organization(s): 
Award Number: 
1119290
Funding Period: 
Thu, 09/01/2011 to Mon, 08/31/2015
Full Description: 

This development and research project from Vanderbilt University, Facet Innovations, and Filament Games, designs, develops, and tests a digital game-based learning environment for supporting, assessing and analyzing middle school students' conceptual knowledge in learning physics, specifically Newtonian mechanics. This research integrates work from prior findings and refines computer assisted testing and Hidden Markov Modeling to develop a new methodology to engage students in deep learning while diagnosing and scaffolding the learning of Newtonian mechanics.

The project uses a randomized experimental 2 x 1 design comparing a single control condition to a single experimental condition with multiple iterations to test the impact of the game on the learning of Newtonian physics. Using designed based research with teachers and students, the researchers are iteratively developing and testing the interactions and knowledge acquisition of students through interviews, pre and post tests and stealth assessment. Student learner action logs are recorded during game-play along with randomized student interviews. Students' explanations and game-play data are collected and analyzed for changes in domain understanding using pre-post tests assessment.

The project will afford the validation of EGAME as an enabler of new knowledge in the fields of cognition, conceptual change, computer adaptive testing and Hidden Markov Modeling as 90 to 300 middle school students learn Newtonian mechanics, and other science content in game-based learning and design. The design of this digital game platform encompasses a very flexible environment that will be accessible to a diverse group of audiences, and have a transformational affect that will advance theory, design and practice in game-based learning environments.

Implementing the Mathematical Practice Standards: Enhancing Teachers' Ability to Support the Common Core State Standards

This is a four-year project that is producing materials designed to help teachers see how the mathematical practices described in the Common Core State Standards for mathematics can be implemented in mathematics instruction. The goal of the improved instruction is to help students adopt and value these critical mathematical practices.

Award Number: 
1119163
Funding Period: 
Mon, 08/01/2011 to Tue, 07/31/2012
Full Description: 

The Implementing Mathematical Practices Standards (IMPS) is a four-year project that is producing materials designed to help teachers see how the mathematical practices described in the Common Core State Standards for mathematics can be implemented in mathematics instruction. The goal of the improved instruction is to help students adopt and value these critical mathematical practices. Researchers at the Education Development Center are developing videos and print materials that exemplify the mathematical practices and are working with teachers in grades 5-10 to help them use the materials effectively. The research questions of the project are focused on what features of the materials are most helpful to teachers and what professional development characteristics facilitate implementation of the mathematics practices in classroom instruction. The external evaluation of the project is being conducted by evaluators at TERC who are looking the process of developing materials and how the materials are used.

The materials will include professionally-produced videos exemplifying a particular mathematical practice being implemented in a classroom as well as printed dialogues that are designed to help teachers understand the practice and why it is critical for students to acquire that mathematical practice. The exemplars of mathematical practices are being developed based on pilot work and systematic advice from mathematicians, mathematics educators and mathematics teachers in grades 5-10. The design process is iterative and materials are refined based on feedback that is received. Facilitators are being prepared to conduct professional development and materials are being tested by more than 150 teachers in a variety of school districts.

Professional groups such as NCTM and NCSM have called for materials that exemplify the CCSS mathematical practices. They have argued that teachers need to understand how these standards can be achieved in classrooms. IMPS systematic effort to design materials that exemplify the standards and to test not only the materials but also the professional development associated with the materials is responding to the national need. The videos and dialogues will be available through broad dissemination.

Pages

Subscribe to Quantitative