Assessment

Pre-service Teachers' Knowledge for Teaching Algebra for Equity in the Middle Grades

This project is using Second Life and other technology to structure carefully planned learning experiences for pre-service teachers. Virtual technologies are used to provide pre-service teachers practice in presenting and assessing problem solving activities in a virtual classroom with diverse populations. Researchers hypothesize that technology enriched strategies have the potential to deepen pre-service teachers' understanding and effectiveness in teaching emerging algebra concepts to diverse student populations.

Project Email: 
Lead Organization(s): 
Award Number: 
1020132
Funding Period: 
Wed, 09/01/2010 to Fri, 08/31/2012
Full Description: 

This research study enriches the education of STEM teachers by using emergent technology to provide pre-service middle school mathematics teachers early teaching experiences that address topics in algebra and equity. In particular, the research team at Texas A&M University Main Campus is using Second Life and other technology to structure carefully planned learning experiences for pre-service teachers. Virtual technologies are used to provide pre-service teachers practice in presenting and assessing problem solving activities in a virtual classroom with diverse populations. Researchers hypothesize that technology enriched strategies have the potential to deepen pre-service teachers' understanding and effectiveness in teaching emerging algebra concepts to diverse student populations. The activities of the study are designed to answer five research questions:

1. How successful are the proposed design strategies for developing technology-rich activities for pre-service teachers?

2. How effective are technology-rich activities in developing pre-service teachers' deep knowledge and understanding of the algebra concepts of variable, change, and operations?

3. How effective are virtual simulations of problem-based teaching of algebra concepts in enhancing pre-service teachers' knowledge and skill in teaching diverse students?

4. What are the trajectories of pre-service teachers' development of knowledge of teaching algebra concepts to diverse students?

5. To what extent can technology-rich activities be effectively implemented in a whole-class setting?

The study is a design experiment using an iterative process to develop prototypes that pre-service teachers employ to become more effective in teaching algebra concepts to diverse student populations. Up to 25 students in a semester are piloting the materials that are continually being refined based on the results of the piloting. Growth in knowledge and practice by students is being measured by assessment instruments and research observations. The Mathematics Teacher Efficacy and Beliefs Instrument is an existing measure. The Knowledge for Teaching Algebra for Equity (KTAE) test is being developed to use as a pre- and posttest in the final phase of the project when students in sections who use and do not use the prototypes are compared. One hundred students are field testing the KTAE. The five member advisory board is being used for guidance and evaluation.

Products from the study will include prototype problem solving and equity activities, a knowledge for teaching algebra for equity (KTAE) assessment, guide to simulation of mathematics students in Second Life, and trajectories of knowledge for teaching algebra for equity. Findings from the study are being disseminated through journals and at professional conferences. This project capitalizes upon the use of state-of-the-art technology to simulate dynamic classroom experiences beyond the scope of prior teacher training programs. Successful implementation could revolutionize teacher training on a broad scale.

Response to Intervention in Mathematics: Beginning Substantive Collaboration between Mathematics Education and Special Education

This project is organizing and hosting a working conference on Response to Intervention (RtI) and related strategies in teaching and assessment in Mathematics. Goals of this work are: To build a community of researchers and practitioners to identify, expand and sustain research needs in this area; to identify and improve the research available related to teaching mathematics within an RtI model; and to develop resources to support teacher's understanding and application of RtI strategies.

Partner Organization(s): 
Award Number: 
1005328
Funding Period: 
Wed, 09/01/2010 to Wed, 02/29/2012
Full Description: 

The National Council of Teachers of Mathematics (NCTM) in collaboration with the Council on Exceptional Children (CEC) is organizing and hosting a focused working conference on Response to Intervention (RtI) and related strategies in teaching and assessment in Mathematics. The ultimate goals of this work are: To build a core community of researchers and practitioners from mathematics education and special education to identify, expand and sustain the research needs in this critical area; to identify and improve the research available related to teaching mathematics within a Response to Intervention model; and to develop professional development resources to support teachers's (pre-service and in-service) understanding and application of research-based RtI strategies in mathematics.

Expected outcomes include: a preliminary analysis of needed research studies; a synthesis of both mathematics education research and special education research around a key question of interest; and examples of content for inclusion in professional development and pre-service teacher education. Results will be disseminated through NCTM and CEC print, web, and conference facilities.

Using Rule Space and Poset-Based Adaptive Testing Methodologies to Identify Ability Patterns in Early Mathematics and Create a Comprehensive Mathematics Ability Test

This project will develop a new assessment for children ages 3-7 to provide teachers with diagnostic information on a child's development of mathematics facility on ten domains such as counting, sequencing, adding/subtracting, and measurement. The Comprehensive Research-based Mathematics Ability (CREMAT) is being developed using innovative psychometric models to reveal information about children on specific attributes for each of the 10 domains.

Project Email: 
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1313695
Funding Period: 
Sat, 09/01/2012 to Wed, 02/28/2018
Full Description: 

A new assessment for children ages 3-7 is being developed to provide teachers with diagnostic information on a child's development of mathematics facility on ten domains such as counting, sequencing, adding/subtracting, and measurement. The Comprehensive Research-based Mathematics Assessment (CREMA) is being developed using innovative psychometric models to reveal information about children on specific attributes for each of the 10 domains. The CREMA will produce information based on carefully developed learning trajectories in a relative short period of time by using computer adaptive testing. The project is guided by two goals: 1) to produce a cognitively diagnostic adaptive assessment that will yield more useful and detailed information about students' knowledge of mathematics than previously possible, and 2) subject the developmental progressions to close cognitive diagnosis using cutting-edge psychometric approaches. An item pool of about 350 items is being developed that can be used to identify the level of understanding children ages 3-7 have on the 10 domains that have been identified as foundational to further learning in mathematics. A research team headed by Dr. Douglas Clements at the University of Buffalo is conducting the development work while being assisted by Dr. Curtis Tatsuoka, a statistician at Case Western Reserve University.

The CREMA is being developed using leading-edge psychometric models based on Q-Matrix theory, rule-state models, and posets. The initial item pool includes items from the REMA, a previously developed instrument based on unidemensional IRT models. New items are being piloted with at least 200 students from a group of a total of 800 students evenly distributed among pre-K to grade 2. The successful items then are used to create the new CREMA. The new assessment is being field tested with 300 children, pre-K to grade 2. A random sample of 50 students (at least 10 from each grade) is being video taped as they work the items. Specific criteria of convergence are being used for feedback on how specific items are performing to meet the required specifications. An external evaluator is auditing the process and is doing spot checks of item codings and other analyses performed.

The main product will be the CREMA that will be made widely available. This instrument using computer adaptive testing will provide teachers with ready information on young children's understanding of critical mathematical ideas. The new psychometric models that will be used and developed to process multiple attributes from individual items will make large strives to move forward the field of mathematics assessment of young children. A publisher has expressed interest to make the assessment widely available that increases the likelihood the assessment will have large impact on early childhood mathematics learning.

This project was previously funded under award # 1019925.

Formative Assessment in Mathematics: Current Status and Guidelines for Future Developments

In response to the increased use of formative assessment practices in California's PK-12 mathematics classrooms, this project will investigate what formative assessments are in use, how practitioners and students are utilizing these assessments, and how they impact performance on summative state assessments. The main outcome of this study will be a set of research-informed and field-tested conclusions, recommendations, guidelines, and tools for the development and use of new or improved PK-12 mathematics formative assessments.

Award Number: 
1020393
Funding Period: 
Wed, 09/01/2010 to Wed, 08/31/2011
Full Description: 

In response to the increased use of formative assessment practices in California's PK-12 mathematics classrooms, the University of California-Davis and Claremont Graduate University join efforts to investigate what formative assessments are in use, how practitioners and students are utilizing these assessments, and how they impact performance on summative state assessments. Four research questions guide this study: (1) What is the current status of formative assessment use in California's PK & K, elementary, middle, and high schools? (2) What are the content and psychometric properties of the current formative assessments? (3) What impact do formative assessment outcomes have on student academic achievement as measured by summative assessments? and (4) Does the use of formative assessment differentially impact performance of subgroups of students, such as ELLs, students of color, economically disadvantaged students, and students with disabilities?

The study consists of three phases and employs a mixed-methods approach to answer the four research questions. To answer research question 1, Phase I administers an online survey to all school principals in California. Frequency and descriptive analyses of data are conducted to obtain an accurate view of the current status of formative assessments. Software is used to qualitatively analyze responses to open-ended questions. Frequencies of categorical responses and mean and standard deviations of Likert-type responses are compared across respondents with different backgrounds and schools with different characteristics. Additional interviews with test publishers, state directors of assessment and research experts will be conducted to round out the perspective. To address research question 2, Phase II conducts a series of psychometric and content analyses of existing assessments to ascertain validity, reliability, and accessibility of assessments. A total of 120 PK-12 schools with the strongest formative assessment instruments and the greatest formative assessment impact on summative assessments participate in these analyses. Sample sizes for this phase, representative of a four-grade span (2nd, 5th, 8th, and 10th grades), were determined through a power analysis for a minimum of 0.25 standard deviation detectable difference with a statistical power of 80% at a 0.05 type I error. Due to the study's focus on traditionally low-performing and underrepresented populations, a stratified random sampling approach in which subgroup identifiers are used as stratification variables is employed. To assess content properties, the study utilizes Webb's (1997) alignment methodology. Reliability of formative assessments is examined using the internal consistency approach (Cronbach alpha). Validity is analyzed through content- and criterion-related validity approaches, including a Multi-Traits/Multi-Methods approach. To answer research questions 3 and 4, a series of path models are created to explore the power of formative assessments in predicting student performance on the State's end-of-year assessments. In addition, an Ex Post Facto design is conducted to investigate whether one or more preexisting conditions have possibly caused subsequent differences in the groups of participants. To document successful practices, the study's Phase III develops case studies of six schools with effective formative assessment practices and demonstrated academic growth using summative measures. Case studies include audiotaped focus group interviews with teachers and administrators, videotaped observations of teacher meetings, and classroom observations.

The main outcome of this study will be a set of research-informed and field-tested conclusions, recommendations, guidelines, and tools for the development and use of new or improved PK-12 mathematics formative assessments. Outcome tools include a blueprint and test specifications based on the California State Mathematics Content Standards for 8th grade, and a field-tested formative assessment prototype focused on key algebra concepts and skills for that grade level. The Center for the Study of Evaluation at the University of California-Los Angeles serves as the project external evaluator.

Using PISA to Develop Activities for Teacher Education (UPDATE)

This project uses items and data from the Program for International Student Assessment (PISA) to develop two kinds of resources for preparation and professional development of secondary mathematics teachers: one in the form of prototype professional learning materials and a second in the form of PISA-based, research-grounded articles written for mathematics teachers and teacher educators. Work on both resources will focus on algebra and quantitative literacy and on factors influencing educational equity.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1019513
Funding Period: 
Wed, 09/01/2010 to Fri, 08/31/2012
Full Description: 

The UPDATE project seeks to enable significant advances in K-12 teacher and student learning of mathematics by using of items and data from the Program for International Student Assessment (PISA) in ways that enhance the work of mathematics teachers and teacher educators. We hypothesize that PISA can be useful to the field in much the same way as the National Assessment of Educational Progress (NAEP), which has long served as a key source of information for the mathematics education community. In contrast to NAEP and TIMSS, the Program for International Student Assessment (PISA) in the area of mathematics has received little or no attention within the U.S. mathematics education community, beyond noting that the performance of U.S. students is mediocre compared to that of students in many other countries in Asia and Europe. A consequence of the lack of attention to PISA in the U.S. is that we have underutilized a potentially valuable source of information for improvement of mathematics education.

In this project we use PISA as a base to develop resources for mathematics educators to use in teacher education settings. One type of resource comes in the form of prototype professional learning materials that provide opportunities for teachers and students to analyze complex mathematical tasks and student responses to those tasks, focusing on both the mathematics entailed in the task and the understandings of mathematics reflected in students’ responses. The materials will be designed to engage teachers in individual and collaborative inquiry aimed at developing their specialized content knowledge and their pedagogical content knowledge. Materials will be field tested in preservice and inservice teacher professional education settings and also shared at regional and national meetings. A second type of resource comes in the form of PISA-based, research-grounded articles written specifically for mathematics teachers and teacher educators and published in journals that reach these audiences. The articles will be informed not only by our experiences in developing and using the prototype materials, but also by the findings of selected secondary analyses of data collected in the 2003 PISA assessment.

Our work is organized around three distinct focus areas: (1) Algebra – a traditional content topic familiar to mathematics teachers that can be approached in a novel way through PISA tasks; (2) Quantitative Literacy – a nontraditional content topic less familiar to mathematics teachers that can be accessed directly through PISA tasks, and (3) Equity – an issue of import to mathematics educators that can be examined carefully using PISA data. In each component our work blends research inquiry and development, integrating the analysis of tasks and data from the PISA mathematics assessment with the creation of prototype teacher education materials and the preparation of PISA-based, research-grounded articles for teachers and teacher educators.

The results of this exploratory study will be disseminated broadly, and they are likely to generate new activity in research and development related to PISA. Mirroring the tradition of the interpretive reports of NAEP results, we will produce PISA-based resources that can have a significant impact on the mathematics education community as teachers, teacher educators, and graduate students examine the materials and reports we produce and use them to improve the quality of teacher and student learning of mathematics.

This exploratory project led by faculty from the University of Michigan uses items and data from the Program for International Student Assessment (PISA) to develop two kinds of resources for preparation and professional development of secondary mathematics teachers. One type of resource comes in the form of prototype professional learning materials that provide opportunities for teachers and students to analyze complex mathematical tasks and student responses to those tasks, focusing on both the mathematics entailed in the task and the understandings of mathematics reflected in students' responses. A second type of resource comes in the form of PISA-based, research-grounded articles written specifically for mathematics teachers and teacher educators. Work on both resources will focus on the critical content areas of algebra and quantitative literacy and on factors influencing educational equity.

The project is driven by the hypothesis that PISA assessment instruments and findings can be useful to teachers in much the way that prior analyses of NAEP frameworks, items, and data have been. To address the first project objective, the research team will use selected PISA items and student responses to those items to design, develop, and test a collection of professional learning tasks that engage mathematics teachers in individual and collaborative inquiry aimed at enhancing their specialized content knowledge and their pedagogical content knowledge. To address the second project objective, the research team will prepare articles for practitioner journals that will be informed by experiences in developing and using the prototype materials, but also by the findings of selected secondary analyses of data collected in the 2003 PISA assessment.

The results of this work will be a collection of resources for use in various teacher preparation and professional development settings to stimulate thinking of secondary mathematics teachers about issues of curriculum content, student learning, teaching, and assessment.

Using Routines as an Instructional Tool for Developing Students' Conceptions of Proof

This project will develop and systematically investigate a teaching model to assist teachers in developing ideas about proof in grades 2-5. The teaching model provides both a tool for learning on the part of elementary teachers and a model of practice from which they can learn as they implement it.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1019482
Funding Period: 
Wed, 09/01/2010 to Fri, 08/31/2012
Project Evaluator: 
Megan Franke
Full Description: 

Developers and researchers at TERC, the Education Development Center, and Mount Holyoke College are participating in the development and systematic investigation of a teaching model to assist teachers in developing ideas about proof in grades 2-5. The teaching model provides both a tool for learning on the part of elementary teachers and a model of practice from which they can learn as they implement it.

The project is a teaching experiment in which the model is iteratively implemented and refined, first with teachers experienced in incorporating ideas about proof into their classroom instruction (Phase 1), then with teachers who are relatively inexperienced, both in their own understanding of proof and in their knowledge of how their students can learn about proof (Phase 2). Research questions focus on developing the components of the model, the learning of teachers as they implement the model, and the learning of students as they engage in the instruction that is guided by the model, with particular attention to students with varied histories of achievement in grade-level work on number and operations.

The expected outcome is a teaching model that can be tested on a larger scale as well as instruments for assessing student learning and teacher understanding of proof. The model includes printed material with descriptions of the routines and instructional sequences, guidelines for implementing each component, and a teaching framework as well as written and video case examples.

Engaging Secondary Students and Teachers Through a Proficiency-Based Assessment and Reassessment of Learning Outcomes (PARLO) System in Mathematics

This project is researching the efficacy of a learning and assessment system that emphasizes students' attaining proficiency or better on a limited set of high value learning objectives in Algebra.

Award Number: 
0918474
Funding Period: 
Thu, 03/01/2012 to Thu, 02/28/2013
Project Evaluator: 
Research for Better Schools
Full Description: 

Using a clustered randomized control trial of 44 secondary schools in the greater Philadelphia area, the project is researching the efficacy of a learning and assessment system that emphasizes students' attaining proficiency or better on a limited set of high value learning objectives in Algebra and Geometry. The study allows for and expects students to resubmit assignments and be reassessed until they achieve proficiency or greater. In this new classroom dynamic, students assume more responsibility for and be active agents in their own learning. For their part, teachers will adopt instructional strategies and techniques that support their students' ongoing and continuous learning, including defining learning outcomes, providing frequent and individualized feedback, and participating in professional development.

The research questions are: 1) Does the use of proficiency-based assessment and reassessment of learning outcomes (PARLO) in Algebra lead to increases in secondary students' achievement and engagement in mathematics? 2) Does PARLO lead to increased student interest in pursuing more complex mathematics or science courses? Two cohorts of ninth grade students are being followed.

Mapping Developmental Trajectories of Students' Conceptions of Integers

This project is using data from interviews with 160 K-12 students and 20 adults to describe common understandings and progressions of development for negative number concepts and operations. The project is motivated by the widely acknowledged finding that students have difficulty mastering key concepts and skills involved in work with integers.

Project Email: 
Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0918780
Funding Period: 
Sat, 08/15/2009 to Sun, 07/31/2011
Project Evaluator: 
West Ed (Juan Carlos Bojorquez)
Full Description: 

The project Mapping Developmental Trajectories of Students' Conceptions of Integers, led by faculty from San Diego State University, is using data from 160 interviews with K-12 students and 20 adults to describe common understandings and progressions of development for negative number concepts and operations. The project is motivated by the widely acknowledged finding that students have difficulty mastering key concepts and skills involved in work with integers.

Two questions frame and guide the proposed research:

* What are students' conceptions of integers and operations on integers?

* What are possible developmental trajectories of students' understandings?

The investigators are seeking answers to those questions through structured interviews with students in elementary grades prior to instruction about negative numbers (Grades 2 and 4), students in middle grades whose formal learning experiences have already included explicit instruction about integers (Grade 7), high school students who are expected to use prior knowledge about integers in more advanced mathematics (Grade 11 PreCalculus and Calculus students), and adults who use integers in their work.

In addition to providing an empirically-based picture of ways that students reason about negative numbers, the project is producing useful interview protocols and a reliable and valid assessment instrument for describing the understanding and skill of students at various stages on such a progression.

Both the characterization of common learning progressions and the assessment instruments will be broadly useful to curriculum and test developers and teachers in K-12 mathematics classrooms.

Developing an Empirically-tested Learning Progression for the Transformation of Matter to Inform Curriculum, Instruction and Assessment Design

A principled framework is created for the development of learning progressions in science that can demonstrate how their use can transform the way researchers, educators and curriculum developers conceptualize important scientific constructs. Using the construct of transformation of matter, which requires understanding of both discrete learning goals and also the connections between them, a hypothetical learning progression is constructed for grades 5-12.

Lead Organization(s): 
Award Number: 
0822038
Funding Period: 
Mon, 09/15/2008 to Fri, 08/31/2012
Full Description: 

A principled framework is created for the development of learning progressions in science that can demonstrate how their use can transform the way researchers, educators and curriculum developers conceptualize important scientific constructs. Using the construct of transformation of matter, which requires understanding of both discrete learning goals and also the connections between them, a hypothetical learning progression is constructed for grades 5-12. Assessments are developed that link to the learning progression and require students to use cognitively challenging activities such as construction of models and scientific explanation to demonstrate their understanding of topics related to transformation of matter. The resultant set of assessment items can be used to place students along the transformation of matter learning progression, regardless of curriculum. The learning progression is empirically tested in grades 6-8 using mainly, but not exclusively, the chemistry units of the IQWST curriculum in a three year longitudinal study that measures the longitudinal progression of students and the cross-sectional development of teachers as they gain experience with the curriculum. The framework developed for creating the tools can inform the learning of other core ideas in science in emergent sciences that are inherently interdisciplinary. Also investigated is the relationship between student and teacher factors and different levels of students' developmental learning.

Iterative Model Building (IMB): A Program for Training Quality Teachers and Measuring Teacher Quality

This project aims to improve professional development programs for pre-service teachers (PSTs) as a way to improve student learning in mathematics and science. PSTs engage in a series of teaching cycles, and then engage in lesson study groups to develop, teach, and analyze a whole-class lesson. The cycle is completed by reexamining students' knowledge in teaching experiments with pairs of students. These teaching cycles are called Iterative Model Building (IMB).

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0732143
Funding Period: 
Wed, 08/15/2007 to Tue, 07/31/2012
Project Evaluator: 
Center for Evaluation and Education Policy

Pages

Subscribe to Assessment