Formative Assessment Delivery System (FADS)

This project addresses the Measurement goal under the Mathematics and Science Education research program. Specifically, we propose developing and refining an assessment development, delivery, scoring, and report-generating system in the area of mathematics, centered on statistics and modeling.  We have been engaged with colleagues at Vanderbilt University in designing a formative assessment system to support (and help evaluate) their innovative curriculum in this area: the Assessing Data Modeling and Statistical Reasoning (ADM) system developed by Rich Lehrer and his colleagues (Lehrer & Schauble, 2007).  This assessment system has been created using the principles of the BEAR Assessment System (BAS; Wilson, 2005), and it and the curriculum it supports is currently being used in several states (WI, AK, TN), and is being adopted into a broader curriculum that is widely used.  The aim of the current project is: (a) to refine a set of software programs that the Berkeley Evaluation and Assessment Research (BEAR) Center has been developing over the last 10 years that support the development, calibration, use and training for the assessment system, and to develop software interconnections among those programs to allow them to operate seamlessly for users whose roles range from assessment developers to teachers to school administrators, to those who provide professional development for teachers; (b) as a first full trial of that software,  to embed the existing ADM materials in the software, construct computer-deliverable and computer-scorable task equivalents of the current  item bank, and develop new computerized reports and support materials for teachers; and (c) to investigate the usefulness of this new software in the context of the ADM curriculum.

The positive effects of innovative assessments are widely acknowledged (Black & Wiliam, 1998), and we are happy that the BAS is seen as one such innovation.  But we are strongly concerned that the good effects that one can find from early-adopters of such innovations will not be sustained unless the considerable burden of teacher scoring of their students’ formative assessments is lightened.  We believe that it is essential that teachers become experts in interpreting their student’s responses to assessments.  But, equally, we see that it is wise to then relieve them of the burden of continual scoring of large amounts of student work.  Hence, the strategy we have adopted is to involve teachers early on in a deep program of professional development that will include close work with curriculum materials, assessments and student responses to assessments (preferably including a large proportion of work from their own students).  However, once teachers have shown their mastery of the role of scorer and interpreter of such student products, we then provide the teacher with computerized assessments that will deliver and score equivalent assessments for their students, and generate rich interpretational materials to help them with diagnosis and planning.  We expect that teachers will still be called upon to evaluate unusual student responses, and also will need to carry out occasional hand-scoring to keep up their mastery and to adapt to innovations in the curriculum.