Quasi-experimental

Further Development and Testing of the Target Inquiry Model for Middle and High School Science Teacher Professional Development (Collaborative Research: Yezierski)

This project scales and further tests the Target Inquiry professional development model. The model involves teachers in three core experiences: 1) a research experience for teachers, 2) materials adaptation, and 3) an action research project. The original program was implemented with high school chemistry teachers, and was shown to result in significant increases, with large effect sizes, in teachers' understanding of science inquiry and quality of instruction, and in science achievement of those teachers' students.

Partner Organization(s): 
Award Number: 
1118749
Funding Period: 
Mon, 08/15/2011 to Wed, 07/31/2013
Full Description: 

This project scales and further tests the Target Inquiry (TI) professional development model. The TI model involves teachers in three core experiences: 1) a research experience for teachers, 2) materials adaptation, and 3) an action research project. The original program was implemented with high school chemistry teachers at Grand Valley State University (GVSU), and was shown to result in significant increases, with large effect sizes, in teachers' understanding of science inquiry and quality of instruction, and in science achievement of those teachers' students. The scale-up and further testing would involve adding physics, biology and geology at Grand Valley State University, and implementing the program at Miami University (MU) with chemistry teachers. Three research questions will be studied:

1) How do the three TI core experiences influence in-service high school science teachers' (i) understanding of the nature of science; (ii) attitudes and beliefs about inquiry instruction; and (iii) classroom instructional methods in the derivatives of the TI model?

2) How does teacher participation in TI affect students' process skills (scientific reasoning and metacognition) and conceptual understanding of science in the derivatives of the TI model?

3) What are the challenges and solutions related to implementing TI in science disciplines beyond chemistry and in other regions?

The research design is quasi-experimental and longitudinal, incorporating implementation with research, and using quantitative and qualitative methods blended in a design research framework. A total of 54 middle and high school science teachers are being recruited for the study. The TI group is completing the TI program (N = 27; 15 at GVSU; 12 at MU) while the comparison group (same sizes and locations) is not. The comparison group is matched according to individual characteristics and school demographics. All teachers are being studied, along with their students, for 4 years (pre-program, post-RET, post-MA, post-AR/post-program). TI teachers are taking 15 credits of graduate level science courses over three years, including summers. Courses include a graduate seminar focused on preparing for the research experience, the research experience in a faculty member's science lab during the summer, application of research to teaching, action research project development, adaptation and evaluation of inquiry-focused curricula, and interpretation and analysis of classroom data from action research. Consistent feedback from professional development, teachers, and evaluation, including the previous implementation, contributes to a design-based approach. Teacher factors being studied include nature of science, inquiry teaching knowledge and beliefs, and quality of inquiry instruction. Student factors being studied include scientific reasoning; metacognition, self-efficacy, and learning processes in science; and content knowledge and conceptual understanding. Only established quantitative and qualitative instruments are being used. Quantitative analysis includes between-group comparisons by year on post-tests, with pre-tests as covariates, and multi-level models with students nested with teachers, and teachers within sites, with the teacher level as the primary unit of change. Trends over time between the treatment and comparison groups are being examined. The evaluation is using a combination of pre/post causal comparative quantitative measures and relevant qualitative data from project leaders and participants, as well as from the comparison group, to provide formative and summative evaluation input.

Outcomes of the project include documentation and understanding of the impacts on science teachers' instruction and student outcomes of research experiences for teachers when they are supported by materials adaptation and action research, and an understanding of what it takes to scale the model to different science disciplines and a different site. The project is also producing a website of instructional materials for middle and secondary science.

Continuous Learning and Automated Scoring in Science (CLASS)

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items into the "Web-based Inquiry Science Environment" (WISE) program.

Award Number: 
1119670
Funding Period: 
Thu, 09/01/2011 to Mon, 08/31/2015
Full Description: 

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items (i.e., short essays, science narratives, concept mapping, graphing problems, and virtual experiments) into the "Web-based Inquiry Science Environment" (WISE) program. WISE is an online science-inquiry curricula that supports deep understanding through visualization of processes not directly observable, virtual experiments, graphing results, collaboration, and response to prompts for explanations. In partnership with Educational Testing Services (ETS), project goals are: (1) to develop five automated inquiry assessment activities that capture students' abilities to integrate their ideas and form coherent scientific arguments; (2) to customize WISE by incorporating automated scores; (3) to investigate how students' systematic feedback based on these scores improve their learning outcomes; and (4) to design professional development resources to help teachers use scores to improve classroom instruction, and administrators to make better informed decisions about teacher professional development and inquiry instruction. The project targets general science (life, physical, and earth) in three northern California school districts, five middle schools serving over 4,000 6th-8th grade students with diverse cultural and linguistic backgrounds, and 29 science teachers. It contributes to increase opportunities for students to improve their science achievement, and for teachers and administrators to make efficient, evidence-based decisions about high-quality teaching and learning.

A key research question guides this effort: How automated scoring of inquiry assessments can increase success for diverse students, improve teachers' instructional practices, and inform administrators' decisions about professional development, inquiry instruction, and assessment? To develop science inquiry assessment activities, scoring written responses include semantic, syntax, and structure of meaning analyses, as well as calibration of human-scored items with a computer-scoring system through the c-rater--an ETS-developed cyber learning technology. Validity studies are conducted to compare automated scores with human-scored items, teacher, district, and state scores, including sensitivity to the diverse student population. To customize the WISE curriculum, the project modifies 12 existing units and develops nine new modules. To design adaptive feedback to students, comparative studies explore options for adaptive guidance and test alternatives based on automated scores employing linear models to compare student performance across randomly assigned guidance conditions; controlling for covariates, such as prior science scores, gender, and language; and grouping comparison studies. To design teacher professional development, synthesis reports on auto-scored data are created to enable them to use evidence to guide curricular decisions, and comments' analysis to improve feedback quality. Workshops, classroom observations, and interviews are conducted to measure longitudinal teachers' change over time. To empower administrators' decision making, special data reports, using-evidence activities, individual interviews, and observation of administrators' meetings are conducted. An advisory board charged with project evaluation addresses both formative and summative aspects.

A research-informed model to improve science teaching and learning at the middle school level through cyber-enabled assessment is the main outcome of this effort. A total of 21 new, one- to three-week duration standards-based science units, each with four or more automatically scored items, serve as prototypes to improve students' performance, teachers' instructional approaches, and administrators' school policies and practices.

CAREER: Mathematics Instruction for English Language Learners (MI-ELL)

This study is investigating the classroom factors and teacher characteristics that contribute to Latino English Language Learners' (ELL) gains in mathematics learning in the eighth grade. In addition to looking for key characteristics that influence mathematics learning, the researchers are measuring teachers' knowledge of mathematics for teaching, quality of instruction, and knowledge about English learners.

Lead Organization(s): 
Award Number: 
1055067
Funding Period: 
Fri, 07/15/2011 to Sat, 06/30/2018
Full Description: 

This study is investigating the classroom factors and teacher characteristics that contribute to Latino English Language Learners' (ELL) gains in mathematics learning in the eighth grade. Researchers are collaborating with two school districts in Texas to investigate teaching practices. The project includes professional development that incorporates successful strategies found from their investigations. In addition to looking for key characteristics that influence mathematics learning, the researchers are measuring teachers' knowledge of mathematics for teaching, quality of instruction, and knowledge about English learners.

The research design of the five-year study is a two-level cluster design in which students are nested within teachers. The goal is to predict English Language Learners' gains in mathematics achievement on standardized tests from the resources used by teachers. Measures of teacher knowledge include the Learning Mathematics for Teaching instrument, TExES Bilingual Education Supplemental 4-8 Representative Exam, and the Quality of Mathematics Instruction instrument. Variables and their interactions are analyzed to understand their relationship with student achievement. The evaluation plan involves both formative and summative components related to conducting the research and offering the associated professional development. The educational plan includes implementing a Mathematics Bilingual Institute that offers practicing teachers a professional development focused on successful classroom practices.

This project has the potential to help educators throughout the United States understand the best practices that promote mathematical learning for Latino ELL students. It can help us understand teacher characteristics that contribute to student learning and ways to help teachers develop those characteristics.

CAREER: Engaging Elementary Students in Data Analysis Through Study of Physical Activities

This project is investigating the learning that can take place when elementary school students are directly involved in the collection, sense-making, and analysis of real, personally-meaningful data sets. The hypotheses of this work are that by organizing elementary statistics instruction around the study of physical activities, students will have greater personal engagement in data analysis processes and that students will also develop more robust understandings of statistical ideas.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1054280
Funding Period: 
Fri, 07/01/2011 to Sat, 06/30/2018
Full Description: 

This CAREER awardee at Utah State University is investigating the learning that can take place when elementary school students are directly involved in the collection, sense-making, and analysis of real, personally-meaningful data sets. The project responds to increasing attention to data collection and analysis in elementary grades and aims to make important contributions to the knowledge base on effective approaches to these topics. The hypotheses of this work are that by organizing elementary statistics instruction around the study of physical activities, students will have greater personal engagement in data analysis processes and that students will also develop more robust understandings of statistical ideas. Students and teachers from fifth grade classrooms from several elementary schools from northern Utah, are participating in the project. This work is co-funded by the EPSCoR program.

Statistics topics include measures of center and variation. Students use pedometers, heart rate monitors, other probeware, and the TinkerPlots software. The research team investigates the influence of personal ownership and relationships to data on students' understanding of learning of elementary statistics concepts and their ability to analyze data. The research involves multi-year clinical interviews and video-recorded classroom design experiments.

Research results are expected to be published in appropriate journals and are expected to be presented at professional meetings. Lesson plans and student instructional materials related to physical activity, measures of center, and data distributions are made available for use in partner elementary schools.

CAREER: Noticing and Capitalizing on Important Mathematical Moments in Instruction

This project investigates the outcomes of a teacher education model designed to foster prospective mathematics teachers' abilities to notice and capitalize on important mathematical moments in instruction. The project engages prospective teachers in research-like analysis of unedited teacher-perspective classroom video early in their teacher education coursework in order to help them learn to identify, assess the mathematical potential of, and respond to important student ideas and insights that arise during instruction.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1052958
Funding Period: 
Fri, 04/15/2011 to Sat, 03/31/2012
Full Description: 

This CAREER awardee at Michigan Technological University is investigating the outcomes of a teacher education model designed to foster prospective mathematics teachers' abilities to notice and capitalize on important mathematical moments in instruction. The researcher engages prospective teachers in research-like analysis of unedited teacher-perspective classroom video early in their teacher education coursework in order to help them learn to identify, assess the mathematical potential of, and respond to important student ideas and insights that arise during instruction.

The research is based on a quasi-experimental design and involves three cohorts of prospective teachers. Practicing teachers from local schools collaborate with the research team. The data collected consists of classroom video. The video is coded and analyzed using Studiocode, which allows for real-time coding and for multiple users to code and annotate video segments.

The research findings are integrated into the institution's teacher education program and are also disseminated more broadly through publication and presentations at professional meetings.

Enhancing Games with Assessment and Metacognitive Emphases (EGAME)

This development and research project designs, develops, and tests a digital game-based learning environment for supporting, assessing and analyzing middle school students' conceptual knowledge in learning physics, specifically Newtonian mechanics. This research integrates work from prior findings to develop a new methodology to engage students in deep learning while diagnosing and scaffolding the learning of Newtonian mechanics.

Lead Organization(s): 
Award Number: 
1119290
Funding Period: 
Thu, 09/01/2011 to Mon, 08/31/2015
Full Description: 

This development and research project from Vanderbilt University, Facet Innovations, and Filament Games, designs, develops, and tests a digital game-based learning environment for supporting, assessing and analyzing middle school students' conceptual knowledge in learning physics, specifically Newtonian mechanics. This research integrates work from prior findings and refines computer assisted testing and Hidden Markov Modeling to develop a new methodology to engage students in deep learning while diagnosing and scaffolding the learning of Newtonian mechanics.

The project uses a randomized experimental 2 x 1 design comparing a single control condition to a single experimental condition with multiple iterations to test the impact of the game on the learning of Newtonian physics. Using designed based research with teachers and students, the researchers are iteratively developing and testing the interactions and knowledge acquisition of students through interviews, pre and post tests and stealth assessment. Student learner action logs are recorded during game-play along with randomized student interviews. Students' explanations and game-play data are collected and analyzed for changes in domain understanding using pre-post tests assessment.

The project will afford the validation of EGAME as an enabler of new knowledge in the fields of cognition, conceptual change, computer adaptive testing and Hidden Markov Modeling as 90 to 300 middle school students learn Newtonian mechanics, and other science content in game-based learning and design. The design of this digital game platform encompasses a very flexible environment that will be accessible to a diverse group of audiences, and have a transformational affect that will advance theory, design and practice in game-based learning environments.

Development of a Cognition-Guided, Formative-Assessment-Intensive, Individualized Computer-Based Dynamic Geometry Learning System for Grades 3-8

This project is focused on creating, testing, refining, and studying a computer-based, individualized, interactive learning system for intermediate/middle school students or by teachers in classrooms. This learning system is called Individualized Dynamic Geometry Instruction and will contain four instructional modules in geometry and measurement that reflect the recommendations of the Common Core State Standards.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1119034
Funding Period: 
Thu, 09/01/2011 to Mon, 08/31/2015
Project Evaluator: 
Jeff Shih
Full Description: 

Developers and researchers at Ohio State University and KCP Technologies are creating, testing, refining, and studying a computer-based, individualized, interactive learning system for intermediate/middle school students that can be used by them independently (online or offline) or by teachers in classrooms. This learning system is called Individualized Dynamic Geometry Instruction (iDGi) and will contain four instructional modules in geometry and measurement that reflect the recommendations of the Common Core State Standards (CCSS). iDGi courseware fully integrates research-based Learning Progressions (LPs) for guiding students' reasoning; formative-assessment linked to LPs; instructional sequencing that interactively adapts to students' locations in LPs; built-in student monitoring, feedback, and guidance; and research-based principles of educational media into the modules. The software platform for iDGi development is an extended version of the dynamic geometry computer environment, The Geometer's Sketchpad.

The development process follows recommendations in Douglas Clements' Curriculum Research Framework and includes sequences of development, trials with students, data collection, and revision. The research and evaluation are based on random assignment of approximately 350 students to treatment and control groups. Achievement data are collected using developer-constructed instruments with items that reflect the mathematics topics in the CCSS. Researchers explore the variability at the student, teacher, and school levels using the appropriate level of hierarchical linear models.

Commercial publishers have expressed strong interest in publishing online and offline computer versions of iDGi, an iPad version of iDGi, an online management system for iDGi, and support materials for users and teachers.

Further Development and Testing of the Target Inquiry Model for Middle and High School Science Teacher Professional Development (Collaborative Research: Herrington)

This project scales and further tests the Target Inquiry professional development model. The scale-up and further testing would involve adding physics, biology and geology at Grand Valley State University, and implementing the program at Miami University with chemistry teachers. The project is also producing a website of instructional materials for middle and secondary science.

Partner Organization(s): 
Award Number: 
1118658
Funding Period: 
Mon, 08/15/2011 to Wed, 07/31/2013
Full Description: 

This project scales and further tests the Target Inquiry (TI) professional development model. The TI model involves teachers in three core experiences: 1) a research experience for teachers (RET), 2) materials adaptation (MA), and 3) an action research (AR) project. The original program was implemented with high school chemistry teachers at Grand Valley State University (GVSU), and was shown to result in significant increases, with large effect sizes, in teachers' understanding of science inquiry and quality of instruction, and in science achievement of those teachers' students. The scale-up and further testing would involve adding physics, biology and geology at GVSU, and implementing the program at Miami University (MU) with chemistry teachers. Three research questions will be studied:

1) How do the three TI core experiences influence in-service high school science teachers' (i) understanding of the nature of science; (ii) attitudes and beliefs about inquiry instruction; and (iii) classroom instructional methods in two new applications of the TI model?

2) How does teacher participation in TI affect students' process skills (scientific reasoning and metacognition) and conceptual understanding of science in two new applications of the TI model?

3) What are the challenges and solutions related to implementing TI in science disciplines beyond chemistry and in other regions?

The research design is quasi-experimental and longitudinal, incorporating implementation with research, and using quantitative and qualitative methods blended in a design research framework. A total of 54 middle and high school science teachers are being recruited for the study. The TI group is completing the TI program (N = 27; 15 at GVSU; 12 at MU) while the comparison group (same sizes and locations) is not. The comparison group is matched according to individual characteristics and school demographics. All teachers are being studied, along with their students, for 4 years (pre-program, post-RET, post-MA, post-AR/post-program). TI teachers are taking 15 credits of graduate level science courses over three years, including summers. Courses include a graduate seminar focused on preparing for the research experience, the research experience in a faculty member's science lab during the summer, application of research to teaching, action research project development, adaptation and evaluation of inquiry-focused curricula, and interpretation and analysis of classroom data from action research. Consistent feedback from professional development providers, other teachers, and evaluation, including comparison with the previous implementation, contributes to a design-based approach. Teacher factors being studied include beliefs about the nature of science, inquiry teaching knowledge and beliefs, and quality of inquiry instruction. Student factors being studied include scientific reasoning; metacognition, self-efficacy, and learning processes in science; and content knowledge and conceptual understanding. Only established quantitative and qualitative instruments are being used. Quantitative analysis includes between-group comparisons by year on post-tests, with pre-tests as covariates, and multi-level models with students nested within teachers, and teachers within sites, with the teacher level as the primary unit of change. Trends over time between the treatment and comparison groups are being examined. The evaluation is using a combination of pre/post causal comparative quantitative measures and relevant qualitative data from project leaders and participants, as well as from the comparison group, to provide formative and summative evaluation input.

Outcomes of the project include documentation and understanding of the impacts on science teachers' instruction and student outcomes of research experiences for teachers when they are supported by materials adaptation and action research, and an understanding of what it takes to scale the model to different science disciplines and a different site. The project is also producing a website of instructional materials for middle and secondary science.

Efficacy Study of Metropolitan Denver's Urban Advantage Program: A Project to Improve Scientific Literacy Among Urban Middle School Students

This is an efficacy study to determine if partnerships among formal and informal organizations demonstrate an appropriate infrastructure for improving science literacy among urban middle school science students. The study aims to answer the following questions: How does participation in the program affect students' science knowledge, skills, and attitudes toward science; teachers' science knowledge, skills, and abilities; and families engagement in and support for their children's science learning and aspirations?

Award Number: 
1020386
Funding Period: 
Wed, 09/15/2010 to Wed, 08/31/2011
Project Evaluator: 
Maggie Miller
Full Description: 

This is an efficacy study through which the Denver Museum of Nature and Science, the Denver Zoo, the Denver Botanic Gardens, and three of Denver's urban school districts join efforts to determine if partnerships among formal and informal organizations demonstrate an appropriate infrastructure for improving science literacy among urban middle school science students. The Metropolitan Denver Urban Advantage (UA Denver) program is used for this purpose. This program consists of three design elements: (a) student-driven investigations, (b) STEM-related content, and (c) alignment of schools and informal science education institutions; and six major components: (a) professional development for teachers, (b) classroom materials and resources, (c) access to science-rich organizations, (d) outreach to families, (e) capacity building and sustainability, and (e) program assessment and student learning. Three research questions guide the study: (1) How does the participation in the program affect students' science knowledge, skills, and attitudes toward science relative to comparison groups of students? (2) How does the participation in the program affect teachers' science knowledge, skills, and abilities relative to comparison groups of teachers? and (3) How do families' participation in the program affect their engagement in and support for their children's science learning and aspirations relative to comparison families?

The study's guiding hypothesis is that the UA Denver program should improve science literacy in urban middle school students measured by (a) students' increased understanding of science, as reflected in their science investigations or "exit projects"; (b) teachers' increased understanding of science and their ability to support students in their exit projects, as documented by classroom observations, observations of professional development activities, and surveys; and (c) school groups' and families' increased visits to participating science-based institutions, through surveys. The study employs an experimental research design. Schools are randomly assigned to either intervention or comparison groups and classrooms will be the units of analysis. Power analysis recommended a sample of 18 intervention and 18 comparison middle schools, with approximately 72 seventh grade science teachers, over 5,000 students, and 12,000 individual parents in order to detect differences among intervention and comparison groups. To answer the three research questions, data gathering strategies include: (a) students' standardized test scores from the Colorado Student Assessment Program, (b) students' pre-post science learning assessment using the Northwest Evaluation Association's Measures for Academic Progress (science), (c) students' pre-post science aspirations and goals using the Modified Attitude Toward Science Inventory, (d) teachers' fidelity of implementation using the Teaching Science as Inquiry instrument, and (e) classroom interactions using the Science Teacher Inquiry Rubric, and the Reformed Teaching Observation protocol. To interpret the main three levels of data (students, nested in teachers, nested within schools), hierarchical linear modeling (HLM), including HLM6 application, are utilized. An advisory board, including experts in research methodologies, science, informal science education, assessment, and measurement oversees the progress of the study and provides guidance to the research team. An external evaluator assesses both formative and summative aspects of the evaluation component of the scope of work.

The key outcome of the study is a research-informed and field-tested intervention implemented under specific conditions for enhancing middle school science learning and teaching, and supported by partnerships between formal and informal organizations.

Cyber-Enabled Learning: Digital Natives in Integrated Scientific Inquiry Classrooms (Collaborative Research: Wang)

This project investigated the professional development needed to make teachers comfortable teaching with multi-user simulations and communications that students use every day. The enactment with OpenSim (an open source, modular, expandable platform used to create simulated 3D spaces with customizable terrain, weather and physics) also provides an opportunity to demonstrate the level of planning and preparation that go into fashioning modules with all selected cyber-enabled cognitive tools framed by constructivism, such as GoogleEarth and Biologica.

Award Number: 
1020091
Funding Period: 
Wed, 09/01/2010 to Wed, 08/31/2011
Project Evaluator: 
HRI
Full Description: 

There is an increasing gap between the assumptions governing the use of cyber-enabled resources in schools and the realities of their use by students in out of school settings. The potential of information and communications technologies (ICT) as cognitive tools for engaging students in scientific inquiry and enhancing teacher learning is explored. A comprehensive professional development program of over 240 hours, along with follow-up is used to determine how teachers can be supported to use ICT tools effectively in classroom instruction to create meaningful learning experiences for students, reducing the gap between formal and informal learning and improve student learning outcomes. In the first year, six teachers from school districts - two in Utah and one in New York - are educated to become teacher leaders and advisors. Then three cohorts of 30 teachers matched by characteristics are provided professional development and field test units over two years in a delayed-treatment design. Biologists from Utah State University and New York College of Technology develop four modules that meet the science standards for both states - the first being changes in the environment. Teachers are guided to develop additional modules. The key technological resource to be used in the project is the Opensimulator 3D application Server (OpenSim), an open source, modular, expandable platform used to create simulated 3D spaces with customizable terrain, weather and physics. 

The research methodology includes the use of the classroom observations using RTOP and Technology Use in Science Instruction (TUSI), selected interviews of teachers and students and validated assessments of student learning. Evaluation, by an external evaluator, assesses the quality of the professional development and the quality of the cyber-enabled learning resources, as well as reviews the research design and implementation. An Advisory Board will monitor the project. 

The project is to determine the professional development needed to make teachers comfortable teaching with multi-user simulations and communications that students use everyday. The enactment with OpenSim also provides an opportunity to demonstrate the level of planning and preparation that go into fashioning modules with all selected cyber-enabled cognitive tools framed by constructivism, such as GoogleEarth and Biologica.

Pages

Subscribe to Quasi-experimental