Instrument

Using Rule Space and Poset-Based Adaptive Testing Methodologies to Identify Ability Patterns in Early Mathematics and Create a Comprehensive Mathematics Ability Test

This project will develop a new assessment for children ages 3-7 to provide teachers with diagnostic information on a child's development of mathematics facility on ten domains such as counting, sequencing, adding/subtracting, and measurement. The Comprehensive Research-based Mathematics Ability (CREMAT) is being developed using innovative psychometric models to reveal information about children on specific attributes for each of the 10 domains.

Project Email: 
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1313695
Funding Period: 
Sat, 09/01/2012 to Wed, 02/28/2018
Full Description: 

A new assessment for children ages 3-7 is being developed to provide teachers with diagnostic information on a child's development of mathematics facility on ten domains such as counting, sequencing, adding/subtracting, and measurement. The Comprehensive Research-based Mathematics Assessment (CREMA) is being developed using innovative psychometric models to reveal information about children on specific attributes for each of the 10 domains. The CREMA will produce information based on carefully developed learning trajectories in a relative short period of time by using computer adaptive testing. The project is guided by two goals: 1) to produce a cognitively diagnostic adaptive assessment that will yield more useful and detailed information about students' knowledge of mathematics than previously possible, and 2) subject the developmental progressions to close cognitive diagnosis using cutting-edge psychometric approaches. An item pool of about 350 items is being developed that can be used to identify the level of understanding children ages 3-7 have on the 10 domains that have been identified as foundational to further learning in mathematics. A research team headed by Dr. Douglas Clements at the University of Buffalo is conducting the development work while being assisted by Dr. Curtis Tatsuoka, a statistician at Case Western Reserve University.

The CREMA is being developed using leading-edge psychometric models based on Q-Matrix theory, rule-state models, and posets. The initial item pool includes items from the REMA, a previously developed instrument based on unidemensional IRT models. New items are being piloted with at least 200 students from a group of a total of 800 students evenly distributed among pre-K to grade 2. The successful items then are used to create the new CREMA. The new assessment is being field tested with 300 children, pre-K to grade 2. A random sample of 50 students (at least 10 from each grade) is being video taped as they work the items. Specific criteria of convergence are being used for feedback on how specific items are performing to meet the required specifications. An external evaluator is auditing the process and is doing spot checks of item codings and other analyses performed.

The main product will be the CREMA that will be made widely available. This instrument using computer adaptive testing will provide teachers with ready information on young children's understanding of critical mathematical ideas. The new psychometric models that will be used and developed to process multiple attributes from individual items will make large strives to move forward the field of mathematics assessment of young children. A publisher has expressed interest to make the assessment widely available that increases the likelihood the assessment will have large impact on early childhood mathematics learning.

This project was previously funded under award # 1019925.

Language-Rich Inquiry Science with English Language Learners (LISELL)

This exploratory study develops and pilot-tests a model for improving science teaching and learning with middle school ELLs. Study goals include: (1) clarifying pedagogical constructs of language-rich science inquiry and the academic language of science and their relationships across the learning contexts of middle school science classrooms, teacher professional development and family science workshops, (2) developing and refining instruments to study these constructs in context, and (3) conducting pilot tests of the model and instruments.

Award Number: 
1019236
Funding Period: 
Sun, 08/15/2010 to Wed, 07/31/2013
Full Description: 

This exploratory study develops, pilot-tests, and refines a model for improving middle school English Language Learners' (ELLs) science learning. The model incorporates two pedagogical constructs (language-rich science inquiry and academic language development); and three learning settings (teacher professional development workshops, middle school science classrooms, and parent-student-teacher science workshops). The specific objectives of the study are: (1) to clarify the two pedagogical constructs and their relationships across the three learning contexts, (2) to develop and refine instruments that will be useful for the study of these constructs in these learning contexts, and (3) to conduct pilot tests of the model and instruments.

The study's development phase consists of the production, adaptation, and pilot testing of instructional strategies for teachers and learning materials for students. Instructional strategies for teachers are centered on three key inquiry practices: (a) coordinating theory and evidence, (b) controlling variables, and (c) cause and effect reasoning across 6th grade earth science, 7th grade life science, and 8th grade physical science. Learning materials for students consist of lessons in a workbook with units highlighting the study of academic language. Also, this phase of the study includes the development of resources to support parents' participation and measurement instruments to gather data during the research phase of the study.

The research phase of the study consists of pilot testing of the model. Two research questions guide the study: (1 What is the value for ELL students, their teachers and their parents of an instructional model that highlights language-rich science inquiry practices and academic language development strategies?; and (2)What is the value for ELL students, their teachers and their parents of an instructional model that is enacted in the contexts of middle school science classrooms, student-parent-teacher science workshops, and teacher professional development workshops? Assuming a quasi-experimental, pretest-posttest design, a power analysis defined a sample size of 1,000 middle school students (800 for the treatment group, and 200 for the control group) in 40 classrooms of three middle schools in the state of Georgia. A total of 12 teachers (8 science teachers and 2 English for Students of Other Languages teachers) were selected using a targeted strategy; and 40 randomly selected parents constitute the remaining population sample. The intervention consists of the use of teacher instructional strategies focused on exploring and elaborating cause-effect relationships, differentiating between evidence and theory, and identifying and controlling variables; students' use of instructional materials on academic language; and exploration of parents' science funds of knowledge. Data gathering strategies employ five instruments: (a) a teacher-focus-group interview protocol, (b) a teacher observation protocol, (c) a parent-student interview protocol, (d) a student academic language writing test, and (e) a student-constructed-response science inquiry test. Data interpretation strategies include qualitative analysis using narrative and semantic structure analysis and statistical analyses. An advisory board and an evaluator conduct the evaluation component of the study, inclusive of formative and summative aspects.

The outcome of this study is a research-informed and field-tested science instructional model focused on the improved learning of ELLs and a set of valid and reliable measuring instruments.

Evaluating the Developing Mathematical Ideas Professional Development Program: Researching its Impact on Teaching and Student Learning

This is a 3.5-year efficacy study of the Developing Mathematical Ideas (DMI) elementary math teacher professional development (PD) program. DMI is a well-known, commercially available PD program with substantial prior evidence showing its impact on elementary teachers' mathematical and pedagogical knowledge. However, no studies have yet linked DMI directly with changes in teachers' classroom practice, or with improved student outcomes in math. This study aims to remedy this gap.

Project Email: 
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1019769
Funding Period: 
Wed, 09/01/2010 to Fri, 08/31/2012
Project Evaluator: 
Bill Nave
Full Description: 

This is a 3.5-year efficacy study of the Developing Mathematical Ideas (DMI) elementary math teacher professional development (PD) program. DMI was developed by staff from Education Development Center (EDC), SummerMath for Teachers, and TERC, the STEM research and development institution responsible for this research. DMI is a well-known, commercially available PD program with substantial prior evidence showing its impact on elementary teachers' mathematical and pedagogical knowledge. However, no studies have yet linked DMI directly with changes in teachers' classroom practice, or with improved student outcomes in math. This study aims to remedy this gap.

The research questions for the study are:

1) Does participation in the Developing Mathematical Ideas (DMI) professional development program lead to increases in reform-oriented teaching?

2) Does participation in DMI lead to increases in students' mathematics learning and achievement, especially in their ability to explain their thinking and justify their answers?

3) What is the process by which a reform-oriented professional development program can influence teaching practice and, thus, student learning? Through what mechanisms does DMI have impact, and with what kinds of support do we see the desired changes on our outcome measures when the larger professional development context is examined?

The dependent variables for this study include a) teachers' pedagogical and mathematics knowledge for teaching; b) the nature of their classroom practice; and c) student learning/ achievement in mathematics.

The study uses experimental and quasi-experimental methods, working with about 195 elementary grades teachers and their students in Boston, Springfield, Leominster, Fitchburg, and other Massachusetts public schools. Volunteer teachers are randomly assigned either to PD with DMI in the first year of the efficacy study, or to a control group that will wait until the second year of the study to receive DMI PD. Both groups of teachers will be followed through two academic years. Analyses use OLS regression, hierarchical modeling, and structural equation modeling, as appropriate, to compare the two groups and to track changes over time. In this way, the project explores several aspects of a conceptual framework hypothesizing relationships among PD, teacher mathematical and pedagogical knowledge, classroom teaching practice, and student outcomes. There are multiple measures of each construct, including video-analysis of teacher practice, and a new video-based measure of teacher knowledge.

The study tests the impact of DMI in a range of districts (large urban, small urban, suburban) serving an ethnically and economically diverse mix of students. It provides much needed, rigorous evidence testing the efficacy of this reform-oriented professional development program. It also directly explores the commonplace theory that teachers' understanding of content and student thinking and their encouragement of rich mathematical discourse for student sense-making lead to improvement on measures of mathematics achievement. Findings from the study are disseminated to both research and practitioner communities. The project provides professional development in mathematics to about 195 teachers to improve their ability to teach important concepts. If the evidence for efficacy is positive, then even larger-scale use of this PD program is likely.

Using Routines as an Instructional Tool for Developing Students' Conceptions of Proof

This project will develop and systematically investigate a teaching model to assist teachers in developing ideas about proof in grades 2-5. The teaching model provides both a tool for learning on the part of elementary teachers and a model of practice from which they can learn as they implement it.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1019482
Funding Period: 
Wed, 09/01/2010 to Fri, 08/31/2012
Project Evaluator: 
Megan Franke
Full Description: 

Developers and researchers at TERC, the Education Development Center, and Mount Holyoke College are participating in the development and systematic investigation of a teaching model to assist teachers in developing ideas about proof in grades 2-5. The teaching model provides both a tool for learning on the part of elementary teachers and a model of practice from which they can learn as they implement it.

The project is a teaching experiment in which the model is iteratively implemented and refined, first with teachers experienced in incorporating ideas about proof into their classroom instruction (Phase 1), then with teachers who are relatively inexperienced, both in their own understanding of proof and in their knowledge of how their students can learn about proof (Phase 2). Research questions focus on developing the components of the model, the learning of teachers as they implement the model, and the learning of students as they engage in the instruction that is guided by the model, with particular attention to students with varied histories of achievement in grade-level work on number and operations.

The expected outcome is a teaching model that can be tested on a larger scale as well as instruments for assessing student learning and teacher understanding of proof. The model includes printed material with descriptions of the routines and instructional sequences, guidelines for implementing each component, and a teaching framework as well as written and video case examples.

CAREER: Supporting Students' Proof Practices Through Quantitative Reasoning in Algebra

The aim of this project is to explore the hypothesis that a curricular focus on quantitative reasoning in middle grades mathematics can enhance development of student skill and understanding about mathematical proof. The project is addressing that hypothesis through a series of studies that include small group teaching experiments with students, professional development work with teachers, and classroom field tests of curricular units that connect quantitative reasoning and proof in algebra.

Award Number: 
1743356
Funding Period: 
Mon, 03/15/2010 to Fri, 06/30/2017
Full Description: 

The aim of this CAREER project led by Amy Ellis at the University of Wisconsin is to explore the hypothesis that a curricular focus on quantitative reasoning in middle grades mathematics can enhance development of student skill and understanding about mathematical proof. The project is addressing that hypothesis through a series of studies that include small group teaching experiments with students, professional development work with teachers, and classroom field tests of curricular units that connect quantitative reasoning and proof in algebra.

Work of the project will produce: (a) insights into ways of unifying two previously disconnected lines of research on quantitative reasoning and proof; (b) models describing realistic ways to support development of students' proof competencies through quantitative reasoning; (c) improvement in students' understanding of algebra through engagement in proof practices based on quantitative reasoning; (d) insights into middle-school students' thinking as they negotiate the transition from elementary to more advanced mathematics; and (e) increased understanding of teachers' knowledge about proof and their classroom practices aimed at helping students progress towards understanding and skill in proof.

This project was previously funded under award #0952415.

Mapping Developmental Trajectories of Students' Conceptions of Integers

This project is using data from interviews with 160 K-12 students and 20 adults to describe common understandings and progressions of development for negative number concepts and operations. The project is motivated by the widely acknowledged finding that students have difficulty mastering key concepts and skills involved in work with integers.

Project Email: 
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0918780
Funding Period: 
Sat, 08/15/2009 to Sun, 07/31/2011
Project Evaluator: 
West Ed (Juan Carlos Bojorquez)
Full Description: 

The project Mapping Developmental Trajectories of Students' Conceptions of Integers, led by faculty from San Diego State University, is using data from 160 interviews with K-12 students and 20 adults to describe common understandings and progressions of development for negative number concepts and operations. The project is motivated by the widely acknowledged finding that students have difficulty mastering key concepts and skills involved in work with integers.

Two questions frame and guide the proposed research:

* What are students' conceptions of integers and operations on integers?

* What are possible developmental trajectories of students' understandings?

The investigators are seeking answers to those questions through structured interviews with students in elementary grades prior to instruction about negative numbers (Grades 2 and 4), students in middle grades whose formal learning experiences have already included explicit instruction about integers (Grade 7), high school students who are expected to use prior knowledge about integers in more advanced mathematics (Grade 11 PreCalculus and Calculus students), and adults who use integers in their work.

In addition to providing an empirically-based picture of ways that students reason about negative numbers, the project is producing useful interview protocols and a reliable and valid assessment instrument for describing the understanding and skill of students at various stages on such a progression.

Both the characterization of common learning progressions and the assessment instruments will be broadly useful to curriculum and test developers and teachers in K-12 mathematics classrooms.

Developing an Empirically-tested Learning Progression for the Transformation of Matter to Inform Curriculum, Instruction and Assessment Design

A principled framework is created for the development of learning progressions in science that can demonstrate how their use can transform the way researchers, educators and curriculum developers conceptualize important scientific constructs. Using the construct of transformation of matter, which requires understanding of both discrete learning goals and also the connections between them, a hypothetical learning progression is constructed for grades 5-12.

Lead Organization(s): 
Award Number: 
0822038
Funding Period: 
Mon, 09/15/2008 to Fri, 08/31/2012
Full Description: 

A principled framework is created for the development of learning progressions in science that can demonstrate how their use can transform the way researchers, educators and curriculum developers conceptualize important scientific constructs. Using the construct of transformation of matter, which requires understanding of both discrete learning goals and also the connections between them, a hypothetical learning progression is constructed for grades 5-12. Assessments are developed that link to the learning progression and require students to use cognitively challenging activities such as construction of models and scientific explanation to demonstrate their understanding of topics related to transformation of matter. The resultant set of assessment items can be used to place students along the transformation of matter learning progression, regardless of curriculum. The learning progression is empirically tested in grades 6-8 using mainly, but not exclusively, the chemistry units of the IQWST curriculum in a three year longitudinal study that measures the longitudinal progression of students and the cross-sectional development of teachers as they gain experience with the curriculum. The framework developed for creating the tools can inform the learning of other core ideas in science in emergent sciences that are inherently interdisciplinary. Also investigated is the relationship between student and teacher factors and different levels of students' developmental learning.

Iterative Model Building (IMB): A Program for Training Quality Teachers and Measuring Teacher Quality

This project aims to improve professional development programs for pre-service teachers (PSTs) as a way to improve student learning in mathematics and science. PSTs engage in a series of teaching cycles, and then engage in lesson study groups to develop, teach, and analyze a whole-class lesson. The cycle is completed by reexamining students' knowledge in teaching experiments with pairs of students. These teaching cycles are called Iterative Model Building (IMB).

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0732143
Funding Period: 
Wed, 08/15/2007 to Tue, 07/31/2012
Project Evaluator: 
Center for Evaluation and Education Policy

Concept Inventories and Chemistry Misconceptions: Chemistry Education Research Doctoral Scholars Program

In response to the critical need for scholars with deep content knowledge in chemistry and the specialized training to conduct CER, this capacity building project prepares scholars whose research marries expertise in instrument design with extensive literature on chemistry misconceptions, resulting in the development of concept inventories as reliable and valid measures of student learning for use by chemistry teachers (both high school and post-secondary) and chemistry education researchers.

Lead Organization(s): 
Award Number: 
0733642
Funding Period: 
Sat, 09/01/2007 to Wed, 08/31/2011
Project Evaluator: 
Dr. Jennifer Lewis, University of South Florida; Dr. Maralee Mayberry, University of South Florida

Scaling Up Mathematics Achievement (SUMA)

This project aims to (1) investigate whether or not it is possible to successfully scale-up and adapt the Capacity Building Systems Model used in the Gadsden Mathematics Initiative and improve mathematics achievement for all students in a larger school district, and (2) replicate success in broadening the participation of underrepresented groups in entering STEM field by closing the achievement gap and raising the achievement level of underrepresented students in mathematics.

Project Email: 
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0733690
Funding Period: 
Sat, 09/01/2007 to Tue, 08/31/2010
Project Evaluator: 
Cori Groth and Cheryl Harris

Pages

Subscribe to Instrument