Assessment

Using Research-Based Formative Assessment to Improve Mathematics Teaching and Learning

This project provides professional development and support for teachers of mathematics in Grades 3-5 and assesses the impacts of the project through a rigorous cluster randomized control trial. The project supports teachers to provide instruction that helps all students reach ambitious academic goals in mathematics.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1316527
Funding Period: 
Wed, 01/01/2014 to Mon, 07/31/2017
Full Description: 

Using Research-Based Formative Assessment to Improve Mathematics Teaching and Learning builds on almost a decade of research and development by the Vermont Mathematics Partnership's Ongoing Assessment Project (OGAP). The project provides professional development and support for teachers of mathematics in Grades 3-5 and assesses the impacts of the project through a rigorous cluster randomized control trial. The project supports teachers to provide instruction that helps all students reach ambitious academic goals in mathematics by: 1) increasing teachers' knowledge of mathematics and of how students learn specific mathematics content, and 2) providing teachers with specific tools and routines for enacting formative assessment and adapting their instruction. The project has three integrated components: 1) professional development and ongoing support in 30 New York City public schools, 2) research on teachers' use of assessment evidence in instruction, and 3) research on student and teacher outcomes.

Helping students deeply understand mathematical concepts requires teachers to become skilled in formative assessment, particularly in the ongoing analysis of evidence in student work when making instructional decisions: moving beyond right and wrong answers into the more important questions of how students think and reason mathematically, where their misconceptions lie, and how they can be addressed instructionally. Yet research shows that teachers struggle with the analytic aspects of formative assessment, and little is known about how teachers use evidence from student work or thinking to improve their instruction. The project addresses both of these concerns by: 1) implementing a rigorous, research-driven approach to formative assessment in 30 schools; and, 2) studying the effects of the intervention in ways that clearly measure impact on teachers and students, including the link between how teachers interpret student work and how they respond instructionally. The creativity and originality of the project lie in its synthesis of a vast body of knowledge about mathematics teaching and learning into a clearly packaged and presented set of tools, routines, and strategies which are directly usable in practice and can dramatically improve the quality of mathematics instruction. The project is organized around the central goal of improving teachers' formative assessment practice, with the research design providing rigorous evidence of project impacts while simultaneously informing the field.

The project will be implemented in a highly diverse school district serving a large number of students from groups traditionally underrepresented in mathematics and the sciences. The formative assessment system developed through this project will ultimately be made available, through a website and multi-media booklets, to all teachers in New York City public schools and across the country. The OGAP formative assessment system will be tied to college and career readiness standards in mathematics rather than a particular curriculum-although it addresses the same content as the major mathematics curricula-which means the materials, knowledge, and strategies will be usable across settings.

The Validity of Technology-Enhanced Assessment in Geometry

This project contributes to the small research base by exploring the validity of Technology-Enhanced Items (TEIs) in the context of elementary geometry. The project addresses three research questions: 1) To what extent are TEIs a valid measurement of geometry standards in the elementary grades?; 2) To what extent do TEIs provide an improved measurement compared to SR items?; and 3) What are the general characteristics of mathematics standards that might be better measured through TEIs?

Lead Organization(s): 
Award Number: 
1316557
Funding Period: 
Thu, 08/01/2013 to Thu, 12/31/2015
Full Description: 

Assessment developers, state departments of education, and national consortia have focused extensive efforts on including Technology-Enhanced Items (TEIs) on summative and formative assessments. TEIs have a number of potential benefits over traditional, selected-response (SR) items, including the potential to measure higher-level constructs, the reduction of the effects of test-taking skills and guessing, the capture of rich diagnostic information, the reduction of cognitive load from non-relevant constructs, and the engaging nature of their design. The first three benefits are true of constructed-response (CR) items, but TEIs have the added benefit of being automatically scored by computer. Despite the potential benefits of TEIs, and the strong push to include these types of items in assessments, there is a death of research on the validity of inferences made by TEIs and on whether TEIs provide improved measurement over traditional item types. The Validity of Technology-Enhanced Assessment in Geometry (VTAG) project contributes to the small research base by exploring the validity of TEIs in the context of elementary geometry.

The project addresses three research questions:

RQ1: To what extent are TEIs a valid measurement of geometry standards in the elementary grades?

RQ2: To what extent do TEIs provide an improved measurement compared to SR items?

RQ3: What are the general characteristics of mathematics standards that might be better measured through TEIs?

To address these research questions, the researchers develop 20 items (ten SR items and 10 TEIs) for each of the seven Common Core State Standards in fourth and fifth grade geometry. The researchers collect validity evidence using expert review, cognitive labs, and classroom administration of the items. The first two research questions are addressed by evaluating the validity of the items based on a variety of sources, including test content, internal structure, the relationship to other variables, and student response processes. To address the third research question, informed by the results of the prior two, the researchers use qualitative analysis to identify common themes of the standards that were identified as being better measured through TEIs.

QuEST: Quality Elementary Science Teaching

This project is examining an innovative model of situated Professional Development (PD) and the contribution of controlled teaching experiences to teacher learning and, as a result, to student learning. The project is carrying out intensive research about an existing special PD summer institute (QuEST) that has been in existence for more than five years through a state Improving Teacher Quality Grants program.

Lead Organization(s): 
Award Number: 
1316683
Funding Period: 
Thu, 08/15/2013 to Mon, 07/31/2017
Full Description: 

The University of Missouri-Columbia is examining an innovative model of situated Professional Development (PD) and the contribution of controlled teaching experiences to teacher learning and, as a result, to student learning. The project is carrying out intensive research about an existing special PD summer institute (QuEST) that has been in existence for more than five years through a state Improving Teacher Quality Grants program. The project will do the following: (1) undertake more in-depth and targeted research to better understand the efficacy of the PD model and impacts on student learning; (2) develop and field test resources from the project that can produce broader impacts; and (3) explore potential scale-up of the model for diverse audiences. The overarching goals of the project are: (a) Implement a high-quality situated PD model for K-6 teachers in science; (b) Conduct a comprehensive and rigorous program of research to study the impacts of this model on teacher and student learning; and (c) Disseminate project outcomes to a variety of stakeholders to produce broader impacts. A comparison of two groups of teachers will be done. Both groups will experience a content (physics) and pedagogy learning experience during one week in the summer. During a second week, one group will experience "controlled teaching" of elementary students, while the other group will not. Teacher and student gains will be measured using a quantitative and qualitative, mixed-methods design.

Science Assessment Planning Among State Teams

This is a three-day conference designed to support the development and use of K-12 formative and summative assessments aligned with the Framework for K-12 Science Education (NRC, 2012).

Lead Organization(s): 
Award Number: 
1316691
Funding Period: 
Thu, 08/15/2013 to Thu, 07/31/2014
Full Description: 

This is a three-day conference designed to support the development and use of K-12 formative and summative assessments aligned with the Framework for K-12 Science Education (NRC, 2012). The focal point of the conference is to build a shared understanding of the instructional and assessment shifts required by this framework, the types of student classroom and assessment work that yields evidence of student competence, and the key considerations in the development of engaging, responsive, and broadly accessible resources.

The conference hosts two integrated events: (1) an Invitational Research Symposium on Science Assessment with assessment and science education specilaists to analyze challenging constructs within the new science education framework from a measurement perspective; and (2) a State Collaboratives on Assessment and Student Standards working session with states' representatives to develop deep understanding of the most challenging and cutting-edge learning goals embedded in the framework. To achieve its purpose, Educational Testing Service joins efforts with the Council of Chief State School Officers, the Council of State Science Supervisors, and the Alliance for Excellent Education.

The conference outcomes are (1) a set of newly conceptualized science tasks (from state science specialists) aligned with the vision of the new science education framework for science proficiency; and (2) a set of templates, tools, and processes that state teams can use in their jurisdictions (a) to conduct capacity-building sessions, and (b) to ensure that resources address the hard-to-measure constructs articulated in the new science education framework.

Learning Algebra and Methods for Proving (LAMP)

This project tests and refines a hypothetical learning trajectory and corresponding assessments, based on the collective work of 50 years of research in mathematics education and psychology, for improving students' ability to reason, prove, and argue mathematically in the context of algebra. The study produces an evidence-based learning trajectory and appropriate instruments for assessing it.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1317034
Funding Period: 
Tue, 10/01/2013 to Wed, 09/30/2015
Full Description: 

The Learning Algebra and Methods for Proving (LAMP) project tests and refines a hypothetical learning trajectory and corresponding assessments, based on the collective work of 50 years of research in mathematics education and psychology, for improving students' ability to reason, prove, and argue mathematically in the context of algebra. The goals of LAMP are: 1) to produce a set of evidence-based curriculum materials for improving student learning of reasoning, proving, and argumentation in eighth-grade classrooms where algebra is taught; 2) to produce empirical evidence that forms the basis for scaling the project to a full research and development project; and 3) to refine a set of instruments and data collection methods to support a full research and development project. LAMP combines qualitative and quantitative methods to refine and test a hypothetical learning trajectory for learning methods of reasoning, argumentation, and proof in the context of eighth-grade algebra curricula. Using qualitative methods and quantitative methods, the project conducts a pilot study that can be scaled up in future studies. The study produces an evidence-based learning trajectory and appropriate instruments for assessing it.

Over the past two decades, national organizations have called for more attention to the topics of proof, proving, and argumentation at all grade levels. However, the teaching of reasoning and proving remains sparse in classrooms at all levels. LAMP will address this critical need in STEM education by demonstrating ways to improve students' reasoning and argumentation skills to meet the demands of college and career readiness.

This project promises to have broad impacts on future curricula in the United States by creating a detailed description of how to facilitate reasoning and argumentation learning in actual eighth-grade classrooms. At present, a comprehensive understanding of how reasoning and proving skills develop alongside algebraic thinking does not exist. Traditional, entirely formal approaches such as two-column proof have not demonstrated effectiveness in learning about proof and proving, nor in improving other mathematical practices such as problem-solving skills and sense making. While several studies, including studies in the psychology literature, lay the foundation for developing particular understandings, knowledge, and skills needed for writing viable arguments and critiquing the arguments of others, a coherent and complete set of materials that brings all of these foundations together does not exist. The project will test the hypothetical learning trajectory with classrooms with high proportions of Native American students.

Misconceptions Oriented Standards-Based Assessment Resource for Teachers of High School Life Science (MOSART HSLS)

This project is developing and validating an assessment instrument that addresses the life sciences for students and teachers in grades 9 through 12 based on the Misconception Oriented Standards-based Assessment Resource for Teachers (MOSART).

Lead Organization(s): 
Award Number: 
1316645
Funding Period: 
Sun, 09/01/2013 to Thu, 08/31/2017
Full Description: 

Researchers in the Harvard Smithsonian Center for Astrophysics are developing and validating an assessment instrument that addresses the life sciences for students and teachers in grades 9 through 12 based on the Misconception Oriented Standards-based Assessment Resource for Teachers (MOSART). The project is developing 400 new test items that are based on core content domains for life science and are aligning these items with the previous National Science Education Standards to provide a connection to the earlier MOSART assessments. The project is also developing and validating two test instruments that address the cross cutting concepts of energy and matter for grades K-12, with a specific focus on flows, cycles and conservation. The new assessments will be made available to other researchers and practitioner through the project website and their on-line assessment system.

The assessment development is based on the process used in prior work that has produced the other MOSART instruments, including design efforts of assessment specialists, teachers, and learning scientists. Pilot items are tested through crowd-sourcing with online adult test takers. Classic test theory techniques, item response theory and Bayesian techniques model the student responses. Outcomes consist of item parameters, test and sub-test characteristics, and predictive linkages among items. A stratified, nationally representative sample of 250 high school biology teachers field test the items with classrooms of students. Descriptive statistics are generated to establish the state of student knowledge, pre-and post-test performance by item and by standard, and teacher knowledge, including the fraction of items for which teachers have correctly identified the most popular wrong answer. Descriptive analyses are followed by hierarchical linear modeling (HLM) of students within classrooms to examine the relationships between student and teacher knowledge. The dependent variables in HLM are student gain scores. Independent variables include teachers' knowledge, and student performance on grade K-8 assessments.

The MOSART instruments have been a strong line of assessment tools that are based on a model of cognition with a strong research base in misconceptions in science education. That research base is only slowly being augmented with a more coherent framework on learning progressions in STEM education, and the MOSART instruments will have the potential for extensive use for the foreseeable future. The grades 9-12 life science instrument based on coupling core ideas with science and engineering practices addresses the gaps in the current MOSART system of assessments. Given the rich literature on misconceptions in life science and the ubiquity of life science as a course at the high school level, the instrument promises to be as useful as the one for K-8 developed with MSP RETA funding. The new instruments on cross-cutting concepts provides a much needed set of assessments for researchers and practitioners, particularly teacher professional development providers. The transition to coupling core content and sciences practices with both the life sciences and the cross-cutting concepts is an opportunity to expand and update the suite of instruments.

Undergraduate Biology Education Research Program

The goals of this nine-week summer program are to develop undergraduates' knowledge and skills in biology education research, encourage undergraduates to pursue doctoral study of biology teaching and learning, expand the diversity of the talent pool in biology education research, strengthen and expand collaborations among faculty and students in education and life sciences, and contribute to the development of theory and knowledge about biology education in ways that can inform undergraduate biology instruction.

Award Number: 
1262715
Funding Period: 
Sun, 09/01/2013 to Wed, 08/31/2016
Full Description: 

The Undergraduate Biology Education Research (UBER) REU Site engages undergraduates in studying important issues specific to the teaching and learning of biology, with mentorship from faculty in the Division of Biological Sciences and the Mathematics and Science Education Department at the University of Georgia. The goals of this nine-week summer program are to develop undergraduates' knowledge and skills in biology education research, encourage undergraduates to pursue doctoral study of biology teaching and learning, expand the diversity of the talent pool in biology education research by strategically recruiting and mentoring underrepresented and disadvantaged students, strengthen and expand collaborations among faculty and students in education and life sciences, and contribute to the development of theory and knowledge about biology education in ways that can inform undergraduate biology instruction.

A programmatic effort to introduce undergraduates to the discipline of biology education research is unprecedented nationwide. Biology education research as a discipline is quite young, and systematic involvement of undergraduates has not been part of the culture or practice in biology or education. UBER aims to promote cultural change that expands the involvement of undergraduates in biology education research and raises awareness among undergraduates that biology teaching and learning are compelling foci for study that can be pursued at the graduate level and via various career paths. UBER utilizes a combined strategy of broad and strategic recruiting to attract underrepresented minority students as well as students who do not have access to biology education research opportunities at their own institutions. Evaluation plans involve tracking UBER participants over time to understand the trajectories of students who complete undergraduate training in biology education research.

Significant co-funding of this project is provided by the Division of Biological Infrastructure in the NSF Directorate for Biological Sciences in recognition of the importance of educational research in the discipline of biology. The Division of Undergraduate Education and the Division of Research on Learning in Formal and Informal Settings also provides co-funding.

Developing Rich Media-Based Materials for Practice-Based Teacher Education

This research and development project is premised on the notion that recent technological developments have made it feasible to represent classroom work in new ways. In addition to watching recorded videos of classroom interactions or reading written cases, teacher educators and teachers can now watch animations and image sequences, realized with cartoon characters, and made to depict activities that happened, or could have happened, in a mathematics classroom.

Award Number: 
1316241
Funding Period: 
Thu, 08/15/2013 to Tue, 07/31/2018
Full Description: 

The 4-year research and development project, Developing Rich Media-based Materials for Practice-based Teacher Education, is premised on the notion that recent technological developments have made it feasible to represent classroom work in new ways. In addition to watching recorded videos of classroom interactions or reading written cases, teacher educators and teachers can now watch animations and image sequences, realized with cartoon characters, and made to depict activities that happened, or could have happened, in a mathematics classroom. Furthermore, teacher educators and teachers can react to such animations or image sequences by making their own depictions of alternative moves by students or teachers in classroom interaction. And all of that can take place in an on-line, cloud-based environment that also supports discussion fora, questionnaires, and the kinds of capabilities associated with learning management systems. Such technologies offer important affordances to teacher educators seeking to provide candidates with course-based experiences that emphasize the development of practice-based skills. The focus of the project is on mathematics teacher education. This joint project of the University of Maryland Center for Mathematics Education and the University of Michigan will produce 6 to 8 field-tested modules for use in different courses that are a part of mathematics teacher preparation programs. The following two-pronged research question will be resolved: What are the affordances and constraints of the modules and the environment as supports for: (1) practice based teacher education and (2) a shift toward blended teacher education?

The project involves the following activities: (1) a teacher education materials development component; (2) a related evaluation component; and (3) two research components. The development phase seeks to develop both the LessonSketch.org platform and six to eight mathematics teacher education modules for use in preservice teacher education programs from around the country. The modules will be written with practice-based teacher education goals in mind and will use the capacities of the LessonSketch.org platform as a vehicle for using rich-media artifacts of teaching with preservice teacher candidates. LessonSketch Teacher Education Research and Development Fellows will be chosen through a competitive application process. They will develop their respective modules along with teams of colleagues that will be recruited to form their inquiry group and pilot the module activities. The evaluation activity will focus on the materials development aspect of the project. Data will be collected by the LessonSketch platform, which includes interviews with Fellows and their teams, perspectives of module writers, descriptive statistics of module use, and feedback from both teacher educator and preservice teacher end-users about the quality of their experiences. The first research activity of the project is design research on the kinds of technological infrastructure that are useful for practice-based teacher education. The PIs will identify tools that teacher educators need and want beyond the current capabilities for web-based support for use of rich media and will produce prototype tools inside the LessonSketch environment to meet these needs. The second research activity of the project will supplement the evaluation activity by examining the implementation of two of the modules in detail. This aspect of the research will examine the goals of the intended curriculum, the proposed modes of media use, the fidelity of the implemented curriculum, and learnings produced by preservice teachers. This research activity will help the field understand the degree to which practice-based teacher education that is mediated by an online access to rich media would be a kind of practice that could be easily incorporated into existing teacher education structures.

The project will produce 6 to 8 LessonSketch modules for use in teacher education classes. Each module will be implemented in at least eight teacher education classes across the country, which means that between 720 and 960 preservice teacher candidates will study the materials. The project aims to shift the field toward practice-based teacher education by supporting university programs to implement classroom-driven activities that will produce mathematics teachers with strong capabilities to teach mathematics effectively and meaningfully.

CAREER: Reciprocal Noticing: Latino/a Students and Teachers Constructing Common Resources in Mathematics

The goal of this project is to extend the theoretical and methodological construct of noticing to develop the concept of reciprocal noticing, a process by which teacher and student noticing are shared. The researcher argues that through reciprocal noticing the classroom can become the space for more equitable mathematics learning, particularly for language learners.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1253822
Funding Period: 
Wed, 05/15/2013 to Mon, 04/30/2018
Full Description: 

The goal of this project is to extend the theoretical and methodological construct of noticing to develop the concept of reciprocal noticing, a process by which teacher and student noticing are shared. The researcher argues that through reciprocal noticing the classroom can become the space for more equitable mathematics learning, particularly for language learners. Thus, the focus of the project is on developing the concept of reciprocal noticing as a way to support better interactions between teachers and Latino/a students in elementary mathematics classrooms.

The project uses a transformative teaching experiment methodology and is guided by the initial conjectures that to make mathematics classrooms intellectually attractive places, Latino/a students and teachers need to learn to develop common resources for teaching and learning mathematics, and that reciprocal noticing as a process supports teachers and students in developing these common resources for teaching and learning mathematics. The project design centers around two research questions:How do teachers and Latino/a students tune to each other's mathematical ideas and explicitly indicate to one another how their ideas are important for discourse that promotes mathematical reasoning in classrooms characterized by reciprocal noticing? What patterns emerge across four classrooms when teachers and Latino/a students engage in reciprocal noticing?

The concept of reciprocal noticing can significantly enhance emerging research in mathematics education about the importance of teacher noticing. Further, this revised concept of noticing can transform mathematics classroom to better support English Language Learners.

The PI will incorporate project findings and videos into methods courses for preservice elementary teachers.

CAREER: Fraction Activities and Assessments for Conceptual Teaching (FAACT) for Students with Learning Disabilities

This project is studying and supporting the development of conceptual understanding of fractions by students with learning disabilities (LD).  Rather than focusing on whether students can or cannot develop conceptual understanding, the project is focused on uncovering the complex understanding students DO have.

Award Number: 
1708327
Funding Period: 
Tue, 07/01/2014 to Sun, 06/30/2019
Project Evaluator: 
Dr. Mary Little
Full Description: 

Dr. Hunt, a former middle school and elementary school mathematics in inclusive settings in a state-demonstration STEM school, works with students deemed to be at risk for mathematics difficulties or labeled as having disabilities. Hunt contends that research and pedagogical practice for children with disabilities should begin from a respect for children's ways of knowing and learning. Rather than focusing on whether students can or cannot develop conceptual understanding, research should attempt to uncover the complex understanding students DO have. She argues that teaching based in learning theory that positions children's learning as adaptation advances reasoning, sense-making, and co-construction of meaning.

The goal of Hunt's project- "CAREER: Fraction Activities and Assessments for Conceptual Teaching (FAACT)"-  is to study and support the development of conceptual understanding of fractions by students with learning disabilities (LD).  Dr. Hunt is re-conceptualizing intensive intervention as children's knowing and learning in "Small Environments". This approach suggests a redirect of research and instructional practice in mathematics for an underserved population of students. The project has the potential to offer a transformative approach to mathematics instruction for students with LD, bringing together expertise on learning disabilities and mathematics education to address an area in which there is very little research. 

The main outcomes of the project include (a) a theory of knowing, learning, and teaching connected to students with LDs in the small environment of supplemental and intensive intervention, (b) selected research-based trajectories specific to the conceptual understandings of fractions evidenced by students with LD presented in case study format, and (c) a set of practices and tools for teaching in the small environment (e.g., explicated knowing and learning framework; a set of learning situations to be used for teaching and/or formative assessment in fraction concepts, and suggestions for instructional decision making to aid teachers in designing student-centered instruction both in small groups and individualized formats).

This project was previously funded under award #1253254 and 1446250.


Project Videos

2019 STEM for All Video Showcase

Title: Fractional Reasoning: Students with Learning Disabilities

Presenter(s): Jessica Hunt, Andy Khounmeuang, Kristi Martin, Blain Patterson, & Juanita Silva


Pages

Subscribe to Assessment