Assessment

Ecology Disrupted: Using Real Scientific Data About Daily Life to Link Environmental Issues to Ecological Processes in Secondary School Science Classrooms (Collaborative Research: Wyner)

We developed and tested two ecology case study units for urban high school students underserved in their connection to nature. The case studies, based on digital media stories about current science produced by the American Museum of Natural History, use current scientific data to link ecological principles to daily life and environmental issues. Preliminary testing results show that treatment students made significantly higher gains than the control students on the project's major learning goals.

Award Number: 
0918629
Funding Period: 
Tue, 09/01/2009 to Tue, 08/31/2010
Full Description: 

We have refined and tested wo case study units on contemporary issues in ecology for urban middle and high school students underserved in their connection to nature. The case studies are based on two Science Bulletins, digital media stories about current science produced by the American Museum of Natural History (AMNH), which use current scientific data to link ecological principles to real-world environmental issues, and to link issues to human daily life. One unit asks the question, ‘How might snowy and icy roads affect Baltimore’s water supply?’ The other asks the question, ’How might being able to drive between Los Angeles and Las Vegas in just four hours put local bighorn sheep at risk?’ The units provide source material and real data for students to investigate these questions, video profiles of scientists that engage students in the science and the research, and the Museum Science Bulletins media for students to analyze and connect the questions to broader ecological principles and issues. We are using these modules to research the question, “Can curricular units that link environmental issues to ecological principles through analysis of real data from published research on the environmental impacts of familiar everyday activities improve student learning of ecological principles, personal and human environmental impacts and the nature of scientific activity?” 

 

Randomized control trials in the classrooms of 40 ninth grade NYC public school teachers are being used to evaluate the efficacy of the modules.  Assessment items from New York State Regents exams were reviewed and new assessment items were developed, field tested, and analyzed for validity and reliability. Students in the experimental and control classrooms were pre- and post-tested using the assessments.  In addition, teachers completed pre-post surveys, and stratified samples of teachers were observed and interviewed. To evaluate the effects of the intervention on student achievement and on instructional practices, descriptive and inferential statistics, including analysis of variance (ANOVA) models are being employed to addressing the core research question about student achievement. ANOVA models are also being used to measure main effects and interactions between the intervention and other variables as they relate to student achievement. Preliminary analysis indicates that treatments students showed signficantly higher gains than control students on learning of three major project learning goals: 1. Understanding of ecological principles in the context of human impact 2. Understanding daily life in the context of human impact 3. Understanding the nature of scientific evidence.

 

Finally, we will apply our evaluation findings from testing the modules to develop a summative module on oyster fishing in the Chesapeake Bay. Also, in order to disseminate the materials online to a national audience, we will develop an online “kit of parts” of module components to enable teachers to create customized modules that target their students' specific instructional needs.

Formative Assessment in the Mathematics Classroom: Engaging Teachers and Students

This project is developing a two-year, intensive professional development model to build middle-grades mathematics teachers’ knowledge and implementation of formative assessment. Using a combination of institutes, classroom practice, and ongoing support through professional learning communities and web-based resources, this model helps teachers internalize and integrate a comprehensive understanding of formative assessment into daily practice.

Project Email: 
Award Number: 
0918438
Funding Period: 
Tue, 09/15/2009 to Tue, 08/31/2010
Project Evaluator: 
Cynthia Char
Full Description: 

Formative Assessment in the Mathematics Classroom: Engaging Teachers and Students (FACETS) 

This project is submitted as a full research and development project that addresses challenge #3, how can the ability of teachers to provide STEM education be enhanced?

The FACETS project will develop a 2-year, intensive professional development model to build middle grades mathematics teachers’ knowledge and implementation of formative assessment. Using a combination of institutes, classroom practice, and ongoing support through professional learning communities and web-based resources, this model will help teachers internalize and integrate a comprehensive understanding of formative assessment into daily practice. As part of the professional development model, we will create a variety of products:

  • a facilitator’s guide describing the components of the professional development model and suggestions for using the model to provide a professional development program,
  • cyberlearning products such as interactive forums and a vetted resource library, and
  • video and other materials for the professional development activities and resource library.

FACETS includes a formative research component centered on the following questions:

1. How do mathematics teachers’ knowledge and practice of formative assessment change as a result of participation in the proposed professional development?

2. What learning trajectory describes teachers’ learning about formative assessment, and what are common barriers to successful implementation?

Reports of research findings will include journal articles on teachers’ learning trajectory for formative assessment and common barriers to successful implementation faced by teachers.

Intellectual merit: Our field work, supported by existing research, has shown that math teachers have difficulty fully implementing formative assessment in their classroom. Existing professional development programs either present a comprehensive understanding without a focus on mathematics, or focus on mathematics but only emphasize some of the critical aspects needed to bring out the full potential of formative assessment. This project will develop a professional development model that a) presents a comprehensive understanding of formative assessment and b) focuses specifically on mathematics. Furthermore, this project proposes to contribute to the field of mathematics teacher education through a deeper insight into mathematics teachers’ learning and practice of formative assessment. This insight can be used by professional developers and teacher educators in mathematics to make decisions that help teachers progress more effectively in their learning. This project brings together a multi-disciplinary team with expertise in formative assessment, professional development, mathematics, mathematics education, and teacher education research.

Broader impacts: We anticipate that the professional development will have an immediate impact on participating teachers, and on their students, as they learn about and implement formative assessment in their classrooms. Individual districts and schools have expressed an interest in the FACETS professional development program. The New Hampshire State Department of Education also indicates support for statewide implementation. In addition, research results regarding teachers’ learning trajectories for formative assessment will be crucial to inform future professional development and teacher education programs, and to help teachers reflect on, and guide, their own learning. Data regarding the major barriers to teachers’ learning of formative assessment will also impact future professional development by identifying issues needing additional focus, as will data regarding the effect on those barriers of factors such as teaching experience and mathematical knowledge for teaching. Finally, as there is a paucity of video and other examples of formative assessment in mathematics classrooms, the resource library will make widely available a sorely needed resource to teachers grappling with understanding and implementing formative assessment in mathematics classrooms in a practical way.

Community for Advancing Discovery Research in Education (CADRE)

CADRE is the resource network that supports researchers and developers who participate in DR K-12 projects on teaching and learning in the science, technology, engineering and mathematics disciplines. CADRE works with projects to strengthen and share methods, findings, results and products, helping to build collaboration around a strong portfolio of STEM education resources, models and technologies. CADRE raises external audiences’ awareness and understanding of the DR K-12 program, and builds new knowledge.

Project Email: 
Award Number: 
1813076
Funding Period: 
Wed, 10/01/2008 to Wed, 09/30/2020
Full Description: 

This project from the Community for Advancing Discovery Research in Education (CADRE) will provide assistance to Discovery Research K-12 (DRK-12) projects in national dissemination of the R&D contributions of the DRK-12 program. This project will strengthen the capacity, advance the research, and amplify the impact of DRK-12 projects and researchers working in the assessment, learning, and teaching strands. Through this effort, CADRE will advance the goals of the DRK-12 program in preK-12 formal STEM education by responding to the continuing need for communication, collaboration, and innovations among DRK-12 awardees and between awardees and the education system.

CADRE's goals to strengthen the capacity, advance the research, and amplify the influence of over 300 active DRK-12 projects and associated researchers are designed to contribute to improvements in preK-12 STEM education. During this project, CADRE will continue its work in three main areas: (1) supporting the DRK-12 community, (2) connecting awardees in support of knowledge generation, and (3) connecting to the larger community of education research, policy, and practice. CADRE seeks to bring together (virtually and in-person) diverse audiences to contribute to and benefit from the work of DRK-12 projects, thereby further increasing engagement in evidence-based education in the STEM disciplines. CADRE will work to ensure that the knowledge and products produced by and with DRK-12 projects are broadly accessible to a varied group of stakeholders. CADRE will disseminate the research, models, resources, and technologies--both within the program and outside--to the broader education practitioner, research, and policymaking communities. In addition, the CADRE Fellows program will support next generation of scholars and increase the capacity of a diverse group of researchers to participate in and contribute to improving education in the STEM disciplines.

CADRE has been funded since 2008 to carry out this work. Learn more about our previous awards:

Award # 1743807 (2017-18)
Staff: Catherine McCulloch,
Principal Investigator; Amy Busey, Co-Principal Investigator; Leana Nordstrom, Project Manager/Director; Derek Riley, Evaluator; Jennifer Stiles, Project Coordinator
Program Director: Robert Ochsendorf

This award extended and enhanced the resource center (titled the Community for Advancing Discovery Research in Education, or CADRE) for the DRK-12 program. CADRE 2018 strengthened the network's virtual presence in order to (a) generate and disseminate knowledge and products that support research, policy, and practice around key issues in STEM education; (b) foster interaction and collaboration across projects to maximize the individual and collective potential of DRK-12 awards; (c) offer targeted professional development activities and resources that support early career researchers and developers; and (d) provide focused outreach and dissemination efforts to the DRK-12 community, other networks, and broader stakeholder audiences.

CADRE brought together (virtually and in-person) diverse audiences to contribute to and benefit from the work of DRK-12 projects, thereby further increasing engagement in evidence-based STEM education. These efforts included interactive webinars, conference presentations, and the 2018 PI Meeting. CADRE also worked closely with two topical groups to advance DRK-12 work on early learning and broadening participation in STEM education. This award expanded upon previous work to support the professional growth of doctoral students, postdoctoral fellows, and other early career researchers, with a focus on broadening participation of individuals underrepresented in STEM. In addition, CADRE worked with awardees to disseminate research, models, resources, and technologies to the broader education practitioner, research, and policymaking communities.

Award # 1650648 (2016-17)
Staff:
Catherine McCulloch, Principal Investigator; Amy Busey, Research Associate; Leana Nordstrom, Project Associate; Derek Riley, Evaluator; Jennifer Stiles, Project Coordinator
Program Director: David Campbell

This award extended and enhanced the resource center (titled the Community for Advancing Discovery Research in Education, or CADRE) for the DRK-12 program. The multi-faceted approach of CADRE 2017 strengthened the network's virtual presence in order to (a) generate and disseminate knowledge and products that support research, policy, and practice around key issues in STEM education; (b) foster interaction and collaboration across projects to maximize the individual and collective potential of DRK-12 awards; (c) offer targeted professional development activities and resources that support early career researchers and developers; and (d) provide focused outreach and dissemination efforts to the DRK-12 community, other networks, and broader stakeholder audiences. The evaluation supported continuous improvement of the network's design and seeks to identify components that have promise for adaptation in future endeavors and by other networks.

Through a variety of online curated resources and interactive events, the project advanced topics of relevance and importance to the DRK-12 community, the National Science Foundation, and society; supported interaction and collaboration among DRK-12 awardees; and facilitated DRK-12 awardee engagement with policy and practice communities. Informed by their expressed interests and needs, this award expanded upon previous work to support the professional growth of early career researchers and developers, with a focus on broadening participation of individuals underrepresented in STEM. The network supported knowledge generation, synthesis, and dissemination with a lens on DRK-12 resources, materials, and tools within and external to the research and development community. The network also contributed to the knowledge base on the design and implementation of networks intending to support knowledge management and collaboration.

Award # 1449550 (2014-16)
Staff:
Catherine McCulloch, Principal Investigator; Barbara Berns, Former Principal Investigator; Amy Busey, Research Associate; Leana Nordstrom, Project Associate; Derek Riley, Evaluator; Jennifer Stiles, Project Coordinator; Brenda Turnbull, Evaluator
Program Director: Karen King

This award continued and enhanced the resource center (titled the Community for Advancing Discovery Research in Education, or CADRE) for the Discovery Research K-12 program. The project built on the experience and expertise that evolved over six years in the development and implementation of CADRE. With this award, CADRE2 worked to maximize the individual and collective potential of DRK-12 awards by fostering collaboration and cross-sharing, and promoting the generation of new knowledge and products. CADRE2 provided technical support to the awardees through communities of practice, a strong virtual presence, and an annual PI meeting; professional growth opportunities targeted particularly to early career researchers and developers; and aggressive outreach and dissemination to the DRK-12 community and beyond. CADRE2 established connections with other networks to leverage each other's strengths and services. This award also focused on support for early career researchers and developers, looking at interests and needs for professional growth. The network also contributed to the knowledge base on capacity building, and provide a lens on dissemination of DRK-12 resources, materials, and tools within and external to the research and development community. The evaluation focused on components that have promise for adaptation by future endeavors and by other networks.

Award # 0822241 (2008-16)
Staff: Barbara Berns, Principal Investigator; Amy Busey, Research Associate; E. Paul Goldenberg, Co-Principal Investigator; Lisa Marco-Bujosa, Research Associate; Alina Martinez, Co-Principal Investigator; Catherine McCulloch, Co-Principal Investigator;Jacqueline Miller, Co-Principal Investigator; Hadley Moore, Evaluator;Leana Nordstrom, Project Associate; Andrea Palmiter, Support Staff; Derek Riley, Discipline Specialist;Greta Shultz, Evaluator; Brenda Turnbull; Discipline Specialist
Program Director: Elizabeth Vanderputten

CADRE carried out the following activities: (a) portfolio assessment to define the projects in terms of composition and major characteristics and identify project needs; (b) synthesis studies to capture a comprehensive view of the portfolio in order to understand the role that the program plays in advancing K-12 student and teacher learning; (c) individual technical support services to project leadership to enhance the rigor of projects; (d) multiple strategies for in-person and virtual technical support and group consultation to PIs based on the principles of commuties of practice; (e) Principal Investigators (PI) meetings, and (f) assistance in disseminating the DRK-12 projects' results and products within the program and throughout the STEM education community.

 
Our resources can be found across the website. Learn about our Early Career work, browse CADRE products, view our Spotlights and Toolkits, and join us for upcoming events.
 

Project Videos

2019 STEM for All Video Showcase

Title: Broadening Participation in PreK-12 STEM Education

Presenter(s): Catherine McCulloch, Malcolm Butler, Cory Buxton, Salvador Huitzilopochtli, Leanne Ketterlin Geller, & Arthur Powell

2018 STEM for All Video Showcase

Title: The Impact of Education Research

Presenter(s): Catherine McCulloch, Hilda Borko, Amy Busey, & Christine Cunningham


Learning to RECAST Students' Causal Assumptions in Science Through Interactive Multimedia Professional Development Tools

This project produced and is testing a website with tools to help teachers identify when students’ science learning may be limited by how they construe the underlying causal structure of the concepts. It demonstrates students’ difficulties and a pedagogical approach to help them recast their explanations to align them with the causal structure in the scientifically accepted explanations. The site focuses on middle school with in-depth examples in density and ecosystems.

Lead Organization(s): 
Award Number: 
0455664
Funding Period: 
Fri, 07/01/2005 to Sat, 04/30/2011
Project Evaluator: 
EDC
Full Description: 

Understanding the nature of causality is critical to learning a range of science concepts from “everyday science” to the science of complexity. The Understandings of Consequence (UC) Project, funded by NSF, established that students hold default assumptions about the nature of causality that hinder their science learning and that curriculum designed to restructure students’ causal assumptions while learning the science leads to deeper understanding. In this project, the UC team and the Science Media Group (SMG) of the Harvard-Smithsonian Center for Astrophysics collaborated in a five-year iterative design process to create interactive, multimedia professional development website. It has tools to guide middle school physics and biology teachers in assessing the structure of their students’ scientific explanations and in using existing curricula and developing their own curriculum to restructure or RECAST students’ understandings to fit with scientifically accepted explanations. It includes a range of formats including: documentary footage of real-life classrooms; interviews with teachers describing challenges and obstacles they faced introducing the curricula, how these were overcome, and, the benefits they obtained from using the materials; comments by students, which demonstrate the wide range of student prior thinking about specific causal forms as embedded in the science concepts; discussion questions, suggested hands-on activities, and short videotaped “content explorations,” examples of student written work and journals; design guides and questions to help teachers understand the features of and how to design RECAST activities, assessments, and assessment rubrics related to causal understanding in science. We are evaluating the site with 60 teachers and are iteratively improving it.

Evaluation of High School Science Courses

This project collects evidence supporting the validity of test instruments and initial characterization of high school teachers' background and use of materials and pedagogies. The project is constructing and validating multiple forms of test instruments that can be used for the evaluation of interventions (e.g. professional development, implementation of new curricula) and the measurement of aspects of teacher knowledge (e.g. subject matter, knowledge of student misconceptions).

Lead Organization(s): 
Award Number: 
0732151
Funding Period: 
Wed, 08/15/2007 to Fri, 07/31/2009

Creation and Dissemination of Upper-elementary Mathematics Assessment Modules

This project has constructed, pilot tested, validated, and is now disseminating assessments of student achievement for use in upper elementary grades.
Lead Organization(s): 
Award Number: 
0831450
Funding Period: 
Fri, 05/01/2009 to Mon, 04/30/2012
Full Description: 
This project has constructed, pilot tested, validated, and is now disseminating assessments of student achievement for use in upper elementary grades. There are four equivalent forms for each of the fourth and fifth grades, with each form covering (1) number and operations, (2) pre-algebra and algebra, and (3) geometry and measurement. Items are based in the literature on student's cognitive growth and are meant to:
  • Represent central ideas in the subject matter;
  • Focus on the meaning of facts and procedures; and
  • Require more complex responses than traditional multiple-choice problems. 
These forms and associated technical materials can be accessed at: http://cepr.harvard.edu/ncte-student-assessments

Helping Teachers to Use and Students To Learn From Contrasting Examples: A Scale-up Study in Algebra I

Several small-scale experimental classroom studies Star and Rittle-Johnson demonstrate the value of comparison in mathematics learning: Students who learned by comparing and contrasting alternative solution methods made greater gains in conceptual knowledge, procedural knowledge, and flexibility than those who studied the same solution methods one at a time. This study will extend that prior work by developing, piloting, and then evaluating the impact of comparison on students' learning of mathematics in a full-year algebra course.

Lead Organization(s): 
Award Number: 
0814571
Funding Period: 
Mon, 09/15/2008 to Tue, 08/31/2010

Pages

Subscribe to Assessment