Quasi-experimental

Further Development and Testing of the Target Inquiry Model for Middle and High School Science Teacher Professional Development (Collaborative Research: Herrington)

This project scales and further tests the Target Inquiry professional development model. The scale-up and further testing would involve adding physics, biology and geology at Grand Valley State University, and implementing the program at Miami University with chemistry teachers. The project is also producing a website of instructional materials for middle and secondary science.

Partner Organization(s): 
Award Number: 
1118658
Funding Period: 
Mon, 08/15/2011 to Wed, 07/31/2013
Full Description: 

This project scales and further tests the Target Inquiry (TI) professional development model. The TI model involves teachers in three core experiences: 1) a research experience for teachers (RET), 2) materials adaptation (MA), and 3) an action research (AR) project. The original program was implemented with high school chemistry teachers at Grand Valley State University (GVSU), and was shown to result in significant increases, with large effect sizes, in teachers' understanding of science inquiry and quality of instruction, and in science achievement of those teachers' students. The scale-up and further testing would involve adding physics, biology and geology at GVSU, and implementing the program at Miami University (MU) with chemistry teachers. Three research questions will be studied:

1) How do the three TI core experiences influence in-service high school science teachers' (i) understanding of the nature of science; (ii) attitudes and beliefs about inquiry instruction; and (iii) classroom instructional methods in two new applications of the TI model?

2) How does teacher participation in TI affect students' process skills (scientific reasoning and metacognition) and conceptual understanding of science in two new applications of the TI model?

3) What are the challenges and solutions related to implementing TI in science disciplines beyond chemistry and in other regions?

The research design is quasi-experimental and longitudinal, incorporating implementation with research, and using quantitative and qualitative methods blended in a design research framework. A total of 54 middle and high school science teachers are being recruited for the study. The TI group is completing the TI program (N = 27; 15 at GVSU; 12 at MU) while the comparison group (same sizes and locations) is not. The comparison group is matched according to individual characteristics and school demographics. All teachers are being studied, along with their students, for 4 years (pre-program, post-RET, post-MA, post-AR/post-program). TI teachers are taking 15 credits of graduate level science courses over three years, including summers. Courses include a graduate seminar focused on preparing for the research experience, the research experience in a faculty member's science lab during the summer, application of research to teaching, action research project development, adaptation and evaluation of inquiry-focused curricula, and interpretation and analysis of classroom data from action research. Consistent feedback from professional development providers, other teachers, and evaluation, including comparison with the previous implementation, contributes to a design-based approach. Teacher factors being studied include beliefs about the nature of science, inquiry teaching knowledge and beliefs, and quality of inquiry instruction. Student factors being studied include scientific reasoning; metacognition, self-efficacy, and learning processes in science; and content knowledge and conceptual understanding. Only established quantitative and qualitative instruments are being used. Quantitative analysis includes between-group comparisons by year on post-tests, with pre-tests as covariates, and multi-level models with students nested within teachers, and teachers within sites, with the teacher level as the primary unit of change. Trends over time between the treatment and comparison groups are being examined. The evaluation is using a combination of pre/post causal comparative quantitative measures and relevant qualitative data from project leaders and participants, as well as from the comparison group, to provide formative and summative evaluation input.

Outcomes of the project include documentation and understanding of the impacts on science teachers' instruction and student outcomes of research experiences for teachers when they are supported by materials adaptation and action research, and an understanding of what it takes to scale the model to different science disciplines and a different site. The project is also producing a website of instructional materials for middle and secondary science.

Efficacy Study of Metropolitan Denver's Urban Advantage Program: A Project to Improve Scientific Literacy Among Urban Middle School Students

This is an efficacy study to determine if partnerships among formal and informal organizations demonstrate an appropriate infrastructure for improving science literacy among urban middle school science students. The study aims to answer the following questions: How does participation in the program affect students' science knowledge, skills, and attitudes toward science; teachers' science knowledge, skills, and abilities; and families engagement in and support for their children's science learning and aspirations?

Award Number: 
1020386
Funding Period: 
Wed, 09/15/2010 to Wed, 08/31/2011
Project Evaluator: 
Maggie Miller
Full Description: 

This is an efficacy study through which the Denver Museum of Nature and Science, the Denver Zoo, the Denver Botanic Gardens, and three of Denver's urban school districts join efforts to determine if partnerships among formal and informal organizations demonstrate an appropriate infrastructure for improving science literacy among urban middle school science students. The Metropolitan Denver Urban Advantage (UA Denver) program is used for this purpose. This program consists of three design elements: (a) student-driven investigations, (b) STEM-related content, and (c) alignment of schools and informal science education institutions; and six major components: (a) professional development for teachers, (b) classroom materials and resources, (c) access to science-rich organizations, (d) outreach to families, (e) capacity building and sustainability, and (e) program assessment and student learning. Three research questions guide the study: (1) How does the participation in the program affect students' science knowledge, skills, and attitudes toward science relative to comparison groups of students? (2) How does the participation in the program affect teachers' science knowledge, skills, and abilities relative to comparison groups of teachers? and (3) How do families' participation in the program affect their engagement in and support for their children's science learning and aspirations relative to comparison families?

The study's guiding hypothesis is that the UA Denver program should improve science literacy in urban middle school students measured by (a) students' increased understanding of science, as reflected in their science investigations or "exit projects"; (b) teachers' increased understanding of science and their ability to support students in their exit projects, as documented by classroom observations, observations of professional development activities, and surveys; and (c) school groups' and families' increased visits to participating science-based institutions, through surveys. The study employs an experimental research design. Schools are randomly assigned to either intervention or comparison groups and classrooms will be the units of analysis. Power analysis recommended a sample of 18 intervention and 18 comparison middle schools, with approximately 72 seventh grade science teachers, over 5,000 students, and 12,000 individual parents in order to detect differences among intervention and comparison groups. To answer the three research questions, data gathering strategies include: (a) students' standardized test scores from the Colorado Student Assessment Program, (b) students' pre-post science learning assessment using the Northwest Evaluation Association's Measures for Academic Progress (science), (c) students' pre-post science aspirations and goals using the Modified Attitude Toward Science Inventory, (d) teachers' fidelity of implementation using the Teaching Science as Inquiry instrument, and (e) classroom interactions using the Science Teacher Inquiry Rubric, and the Reformed Teaching Observation protocol. To interpret the main three levels of data (students, nested in teachers, nested within schools), hierarchical linear modeling (HLM), including HLM6 application, are utilized. An advisory board, including experts in research methodologies, science, informal science education, assessment, and measurement oversees the progress of the study and provides guidance to the research team. An external evaluator assesses both formative and summative aspects of the evaluation component of the scope of work.

The key outcome of the study is a research-informed and field-tested intervention implemented under specific conditions for enhancing middle school science learning and teaching, and supported by partnerships between formal and informal organizations.

Cyber-Enabled Learning: Digital Natives in Integrated Scientific Inquiry Classrooms (Collaborative Research: Wang)

This project investigated the professional development needed to make teachers comfortable teaching with multi-user simulations and communications that students use every day. The enactment with OpenSim (an open source, modular, expandable platform used to create simulated 3D spaces with customizable terrain, weather and physics) also provides an opportunity to demonstrate the level of planning and preparation that go into fashioning modules with all selected cyber-enabled cognitive tools framed by constructivism, such as GoogleEarth and Biologica.

Award Number: 
1020091
Funding Period: 
Wed, 09/01/2010 to Wed, 08/31/2011
Project Evaluator: 
HRI
Full Description: 

There is an increasing gap between the assumptions governing the use of cyber-enabled resources in schools and the realities of their use by students in out of school settings. The potential of information and communications technologies (ICT) as cognitive tools for engaging students in scientific inquiry and enhancing teacher learning is explored. A comprehensive professional development program of over 240 hours, along with follow-up is used to determine how teachers can be supported to use ICT tools effectively in classroom instruction to create meaningful learning experiences for students, reducing the gap between formal and informal learning and improve student learning outcomes. In the first year, six teachers from school districts - two in Utah and one in New York - are educated to become teacher leaders and advisors. Then three cohorts of 30 teachers matched by characteristics are provided professional development and field test units over two years in a delayed-treatment design. Biologists from Utah State University and New York College of Technology develop four modules that meet the science standards for both states - the first being changes in the environment. Teachers are guided to develop additional modules. The key technological resource to be used in the project is the Opensimulator 3D application Server (OpenSim), an open source, modular, expandable platform used to create simulated 3D spaces with customizable terrain, weather and physics. 

The research methodology includes the use of the classroom observations using RTOP and Technology Use in Science Instruction (TUSI), selected interviews of teachers and students and validated assessments of student learning. Evaluation, by an external evaluator, assesses the quality of the professional development and the quality of the cyber-enabled learning resources, as well as reviews the research design and implementation. An Advisory Board will monitor the project. 

The project is to determine the professional development needed to make teachers comfortable teaching with multi-user simulations and communications that students use everyday. The enactment with OpenSim also provides an opportunity to demonstrate the level of planning and preparation that go into fashioning modules with all selected cyber-enabled cognitive tools framed by constructivism, such as GoogleEarth and Biologica.

Evaluating the Developing Mathematical Ideas Professional Development Program: Researching its Impact on Teaching and Student Learning

This is a 3.5-year efficacy study of the Developing Mathematical Ideas (DMI) elementary math teacher professional development (PD) program. DMI is a well-known, commercially available PD program with substantial prior evidence showing its impact on elementary teachers' mathematical and pedagogical knowledge. However, no studies have yet linked DMI directly with changes in teachers' classroom practice, or with improved student outcomes in math. This study aims to remedy this gap.

Project Email: 
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1019769
Funding Period: 
Wed, 09/01/2010 to Fri, 08/31/2012
Project Evaluator: 
Bill Nave
Full Description: 

This is a 3.5-year efficacy study of the Developing Mathematical Ideas (DMI) elementary math teacher professional development (PD) program. DMI was developed by staff from Education Development Center (EDC), SummerMath for Teachers, and TERC, the STEM research and development institution responsible for this research. DMI is a well-known, commercially available PD program with substantial prior evidence showing its impact on elementary teachers' mathematical and pedagogical knowledge. However, no studies have yet linked DMI directly with changes in teachers' classroom practice, or with improved student outcomes in math. This study aims to remedy this gap.

The research questions for the study are:

1) Does participation in the Developing Mathematical Ideas (DMI) professional development program lead to increases in reform-oriented teaching?

2) Does participation in DMI lead to increases in students' mathematics learning and achievement, especially in their ability to explain their thinking and justify their answers?

3) What is the process by which a reform-oriented professional development program can influence teaching practice and, thus, student learning? Through what mechanisms does DMI have impact, and with what kinds of support do we see the desired changes on our outcome measures when the larger professional development context is examined?

The dependent variables for this study include a) teachers' pedagogical and mathematics knowledge for teaching; b) the nature of their classroom practice; and c) student learning/ achievement in mathematics.

The study uses experimental and quasi-experimental methods, working with about 195 elementary grades teachers and their students in Boston, Springfield, Leominster, Fitchburg, and other Massachusetts public schools. Volunteer teachers are randomly assigned either to PD with DMI in the first year of the efficacy study, or to a control group that will wait until the second year of the study to receive DMI PD. Both groups of teachers will be followed through two academic years. Analyses use OLS regression, hierarchical modeling, and structural equation modeling, as appropriate, to compare the two groups and to track changes over time. In this way, the project explores several aspects of a conceptual framework hypothesizing relationships among PD, teacher mathematical and pedagogical knowledge, classroom teaching practice, and student outcomes. There are multiple measures of each construct, including video-analysis of teacher practice, and a new video-based measure of teacher knowledge.

The study tests the impact of DMI in a range of districts (large urban, small urban, suburban) serving an ethnically and economically diverse mix of students. It provides much needed, rigorous evidence testing the efficacy of this reform-oriented professional development program. It also directly explores the commonplace theory that teachers' understanding of content and student thinking and their encouragement of rich mathematical discourse for student sense-making lead to improvement on measures of mathematics achievement. Findings from the study are disseminated to both research and practitioner communities. The project provides professional development in mathematics to about 195 teachers to improve their ability to teach important concepts. If the evidence for efficacy is positive, then even larger-scale use of this PD program is likely.

Effective Programs for Elementary Science: A Best-evidence Synthesis

This synthesis project is a systematic review of experimental research evaluating programs and practices in elementary science. The systematic review addresses all areas of science in the elementary grades. The review uses an adaptation of best-evidence synthesis previously applied to elementary and secondary mathematics and reading, and includes experimental and quasi-experimental research on the outcomes of alternative approaches to elementary science.

Lead Organization(s): 
Award Number: 
1019306
Funding Period: 
Wed, 09/01/2010 to Fri, 08/31/2012
Full Description: 

This synthesis project is a systematic review of experimental research evaluating programs and practices in elementary science. The systematic review addresses all areas of science in the elementary grades. Different versions of the synthesis are written for audiences of researchers, policy makers, principals, and teachers. The review uses an adaptation of best-evidence synthesis previously applied to elementary and secondary mathematics and reading, and includes experimental and quasi-experimental research on the outcomes of alternative approaches to elementary science. The review is a part of a series of reviews that are part of the Best Evidence Encyclopedia (BEE), an on-line resource that disseminates systematic reviews of research on achievement outcomes of programs at all subject areas and grade levels (see www.bestevidence.org), and is led by Robert Slavin of Johns Hopkins University.

The review is carried out by a US-UK partnership of science educators and experts on systematic reviews of research. An advisory group of scientists, science educators, and experts on research review oversees the design of the review, monitors review procedures, and comments on drafts. This review takes a broad approach to searching the literature in order to locate every study that meets inclusion requirements for valid research. It includes electronic searches of educational databases (JSTOR, ERIC, EBSCO, Psych INFO, Dissertation Abstracts) using different combinations of key words (for example, "elementary students" and "science achievement"), covering the years 1970-2010. Results are narrowed by subject area (for example, "educational software", "science achievement", "instructional strategies"). Web-based repositories and education publishers' websites are included. The review also discusses each study that meets the inclusion requirements for a valid research design.

A strength of this work is that it takes on the synthesis of what is known about best practice for elementary science education, relying only on studies that meet the criteria for inclusion as having credible research designs. This is a review that is sorely needed in the field of science education. The lengthy and detailed review will be available on the BEE network, along with educator-friendly summaries. The work is also vetted via publication in a top, peer-reviewed journal. The study will include a set of tables showing ratings of programs according to consistent criteria in terms of the strength of the evidence base for each, with brief descriptions of the methods and findings. This educators' summary, patterned on Consumer Reports, is intended primarily for superintendents, principals, and teachers who are making choices among programs for implementation with their children.

DRK12-Biograph: Graphical Programming for Constructing Complex Systems Understanding in Biology

This project will investigate how complex systems concepts supported by innovative curricular resources, technology applications and a comprehensive research and development structure can assist student learning in the domain of biology by providing a unifying theme across scales of time and space. The project seeks to address four areas of critical need in STEM education: biological sciences, complex systems, computational modeling, and equal access for all.

Award Number: 
1019228
Funding Period: 
Wed, 09/01/2010 to Sun, 08/31/2014
Project Evaluator: 
David Reider
Full Description: 

This proposal outlines a research and development project that investigates how complex systems concepts supported by innovative curricular resources, technology applications and a comprehensive research and development structure can assist student learning in the domain of biology by providing a unifying theme across scales of time and space. The project seeks to address four areas of critical need in STEM education: biological sciences, complex systems, computational modeling, and equal access for all. This proposal explores how these needs are addressed through a curricular and technological intervention that structures biology learning through the framework of complex systems and computational modeling. The primary partners are the Massachusetts Institute of Technology and the University of Pennsylvania, working with eight teachers in four schools in the Boston area.

The project integrates graphical programming and simulation software, StarLogo TNG, into the standard high school biology curriculum to improve learning of biology concepts through the introduction and understanding of core complex systems processes. Instead of learning biology in discrete chunks, the chosen biological topics are connected through the framework of complex systems, and successively build in complexity from the basic building blocks of life to the interdependence and sustainability of life forms. This approach is designed to help students understand how processes at one level are connected to those at another level. The research is designed to answer the following questions: 1. Does a learning progression based on the complex systems ideas of scale and emergence enable students to make connections across biological topics, remediate known misconceptions, and apply core complex systems principles better than traditional instructional sequences? 2. What are the on-going affordances and constraints of implementation taking into consideration structural, functional and behavioral variables and what changes to project activities yield increased implementation and learning capacities? 3. Does programming of simulations increase understanding of complex systems and biology concepts compared to use of previously constructed simulations? The evaluation is designed to collect data and provide feedback on the adherence to the plan, the implementation challenged, and how research informs development.

The project anticipates a number of deliverables towards the end of the project and beyond. These include the creation of a unified high school biology curricular sequence that builds in increasing spatial and temporal scales to deepen student understanding of four core biology topics; the production, implementation and testing of curricular activities that acknowledge and ameliorate known implementation challenges; and the development of curricular strategies and tools to help teachers and students improve knowledge and skills in computational modeling, computer programming and participation in the cyberinfrastructure. In order to increase ease of integration into schools, and enhance scalability, the simulation activities are facilitated by a new web-based version of StarLogo TNG that integrates the curricular materials all of which will be distributed freely. Additional dissemination strategies include a website, conferences, a newsletter, community activities, active dissemination, and academic presentations.

Data Explorations in Ecology Project (DEEP)

This project evaluates the benefits of using different types of place-based ecological data in high school science classrooms. This project will assess the use of first-hand (collected by students) and real-time second-hand data in teaching science and critical thinking skills. The guiding question for the project is "Does using place-based, first-hand ecological evidence, and relating that to place-based, second-hand data, improve students' environmental science literacy, nature of science understanding, and knowledge of ecological concepts?"

Partner Organization(s): 
Award Number: 
1020186
Funding Period: 
Wed, 09/01/2010 to Sat, 08/31/2013
Full Description: 

This project evaluates the benefits of using different types of place-based ecological data in high school science classrooms. This project will combine and assess the use of first-hand (collected by students) and real-time second-hand data in teaching science and critical thinking skills. The guiding question for the project is "Does using place-based, first-hand ecological evidence, and relating that to place-based, second-hand data, improve students' environmental science literacy, nature of science understanding, and knowledge of ecological concepts?" Other questions the proposed project will explore include: How can teachers best engage students in understanding and evaluating critical environmental problems through the use of data? Does the use of real-time data in the classroom help connect students with science content and/or the scientific research community? What knowledge and skills do teachers need in order to make effective use of the data being made available to them by ecological monitoring networks such as National Ecological Observatory Network (NEON)?

To answer these questions, a place-based, ecology curriculum, the Changing Hudson Project, will be used along with data and field trips provided by regional partners. A quasi-experimental study in high school classes in the Mid-Hudson Valley of NY will compare different instructional models, providing preliminary evidence of the relative strengths and limitations of different approaches. A range of formative assessment methods will be used to describe and assess students' understanding of ecology, and their engagement, motivation and capacity for collecting, analyzing, and applying ecological data. The evaluation will include pre-and post-assessments given to students in the treatment classrooms and in a comparison classroom in the same schools. Questionnaires, focus group interviews, and student portfolios will be used to assess student understanding and dispositions in sample classrooms.

This proposal addresses an exciting and interesting area of research regarding the inquiry approach to science and the utility of cyber-enabled science investigations. Many K-12 teachers find it difficult to expose students to the real environment. Field trips can be expensive, and liability concerns scare many teachers and especially school administrators away from allowing students to experience natural settings outside of the classroom. This phenomenon is lamented by ecologists and has led to a movement to get kids outside more. The concern is that students today have a 'Nature Deficit Disorder,' as coined by Richard Louv in his book 'Last Child in the Woods.' Advocates of cyber-learning propose that technology provides a solution by allowing students to experience the outside world virtually, and that they can collect and analyze ecological data from the comfort of their classroom desks. Virtual experiences may be better than no experience at all, but how do they compare with first-hand experiences? This proposal aims to determine how virtual experiences compare to real-life experiences with regard to understanding ecological concepts, analyzing ecological data, and drawing scientifically-reasoned, valid conclusions.

Assessing Instructional Quality in Mathematics: A Comparative Study of High and Low Value-Added Teachers' Videotaped Lessons

This project’s researchers are determining individual teacher effect estimates and investigating their stability across models. This study also investigates the instructional practices of a subsample of 30 highly effective and 30 less effective sixth-grade mathematics teachers using videotaped classroom lessons, which are coded and analyzed by researchers who are blind to the value-added effectiveness of the teachers.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0949241
Funding Period: 
Mon, 10/01/2007 to Mon, 08/31/2009

International Workshop on Mathematics and Science Education: Common Priorities that Promote Collaborative Research

The goal of this workshop is to advance the construction of new knowledge through international cooperation with Chinese counterparts in the teaching and learning of math and science at the elementary level in four areas: curriculum design and assessment; teacher preparation and professional development; effective use of the former; and reaching gifted and underserved populations. Approximately 120 people will attend, including 50 senior U.S. researchers, 25 early career researchers, 15 graduate students and 5 undergraduates.

Award Number: 
0751664
Funding Period: 
Sat, 03/15/2008 to Mon, 02/28/2011

Engaging Youth in Engineering Module Study

This project is revising and field testing six existing modules and developing, pilot testing, and field testing two engineering modules for required middle school science and mathematics classes: Catch Me if You Can! with a focus on seventh grade life science; and Creating Bioplastics targeting eighth grade physical science. Each module addresses an engineering design challenge of relevance to industries in the region and fosters the development of engineering habits of mind.

Award Number: 
0918769
Funding Period: 
Tue, 09/15/2009 to Sun, 08/31/2014
Project Evaluator: 
James Van Haneghan

Pages

Subscribe to Quasi-experimental