Middle School

Formative Assessment Delivery System (FADS)

This project uses new psychometric techniques to create a technological tool that could evaluate how well students in the 4th-8th mathematics and science classrooms respond to complex performance tasks. The purpose of this tool is to improve the instruction of teachers in mathematics and science. It will produce real-time individualized diagnoses of instructional needs to help teachers plan instruction that specifically addresses the learning needs of each student in that class.

Project Email: 
Award Number: 
0733334
Funding Period: 
Wed, 08/15/2007 to Sat, 07/31/2010
Project Evaluator: 
Julie Koppich
Full Description: 

This project addresses the Measurement goal under the Mathematics and Science Education research program. Specifically, we propose developing and refining an assessment development, delivery, scoring, and report-generating system in the area of mathematics, centered on statistics and modeling.  We have been engaged with colleagues at Vanderbilt University in designing a formative assessment system to support (and help evaluate) their innovative curriculum in this area: the Assessing Data Modeling and Statistical Reasoning (ADM) system developed by Rich Lehrer and his colleagues (Lehrer & Schauble, 2007).  This assessment system has been created using the principles of the BEAR Assessment System (BAS; Wilson, 2005), and it and the curriculum it supports is currently being used in several states (WI, AK, TN), and is being adopted into a broader curriculum that is widely used.  The aim of the current project is: (a) to refine a set of software programs that the Berkeley Evaluation and Assessment Research (BEAR) Center has been developing over the last 10 years that support the development, calibration, use and training for the assessment system, and to develop software interconnections among those programs to allow them to operate seamlessly for users whose roles range from assessment developers to teachers to school administrators, to those who provide professional development for teachers; (b) as a first full trial of that software,  to embed the existing ADM materials in the software, construct computer-deliverable and computer-scorable task equivalents of the current  item bank, and develop new computerized reports and support materials for teachers; and (c) to investigate the usefulness of this new software in the context of the ADM curriculum.

The positive effects of innovative assessments are widely acknowledged (Black & Wiliam, 1998), and we are happy that the BAS is seen as one such innovation.  But we are strongly concerned that the good effects that one can find from early-adopters of such innovations will not be sustained unless the considerable burden of teacher scoring of their students’ formative assessments is lightened.  We believe that it is essential that teachers become experts in interpreting their student’s responses to assessments.  But, equally, we see that it is wise to then relieve them of the burden of continual scoring of large amounts of student work.  Hence, the strategy we have adopted is to involve teachers early on in a deep program of professional development that will include close work with curriculum materials, assessments and student responses to assessments (preferably including a large proportion of work from their own students).  However, once teachers have shown their mastery of the role of scorer and interpreter of such student products, we then provide the teacher with computerized assessments that will deliver and score equivalent assessments for their students, and generate rich interpretational materials to help them with diagnosis and planning.  We expect that teachers will still be called upon to evaluate unusual student responses, and also will need to carry out occasional hand-scoring to keep up their mastery and to adapt to innovations in the curriculum.

Validation and Refinement of a Model for Teacher Professional Development that Leverages a Major Applied Research Laboratory

This project focuses on the assessed impact of a teacher professional development (TPD) program around the applied sciences. Specifically, researchers seek to examine the measurable impact of an established teacher professional development program currently offered through the UC Davis Edward Teller Education Center. The Center delivers teacher training and curricula and draws upon an instructor cohort that pairs a regional master teacher and relevant LLNL scientists in curriculum development and delivery.

Award Number: 
0733350
Funding Period: 
Wed, 08/15/2007 to Sat, 07/31/2010

Capacity Building Conference Series: Supporting an Emerging Community of Science Education Researchers

This project covers participants' costs to attend a national conference series focusing upon supporting incipient science education research projects. A primary objective is to provide a venue in which researchers can describe their lines of inquiry and to then receive guidance and input about refining those ambitions. The other primary objective is to promote an innovative conference design in which a structured presentation format serves as an incubator for scholarly work.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0711264
Funding Period: 
Wed, 08/01/2007 to Sat, 07/31/2010

Design and Use of Illustrations in Test Items as a Form of Accommodation for English Language Learners in Science and Mathematics Assessment

This project investigates how vignette illustrations minimize the impact of limited English proficiency on student performance in science tests. Different analyses will determine whether and how ELL and non-ELL students differ significantly on the ways they use vignettes to make sense of items; whether the use of vignettes reduces test-score differences due to language factors between ELL and non-ELL students; and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0822362
Funding Period: 
Mon, 09/01/2008 to Sun, 08/12/2012
Full Description: 

This exploratory project within the Contextual Strand (Challenge a) addresses validity in the assessment of science and mathematics for English language learners (ELLs), and the urgent need for effective testing accommodations for ELLs. Motivation for this investigation originated from a previous, NSF-funded project on the testing of ELLs. We observed that items which were accompanied by illustrations tended to be responded correctly by a higher percentage of students than items without illustrations. We will investigate the factors that are relevant to designing and using a new form of accommodation in the assessment of science and mathematics for ELLs—vignette illustrations.

This three-year project will be guided by four research questions: What principles underlie the effective design of science and mathematics test items with illustrations in ways that minimize limited English proficiency as a factor that prevents ELLs from understanding the items? Is the presence of an illustration a moderator in students’ understanding test items? If so, Is the effect due to the simple presence of a graphical component or due to characteristics of the illustrations that are created based on principled design? Does the presence of an illustration have a different effect on the performance of ELLs and the performance of non-ELL students?

We expect to be able to: 1) identify the role of illustrations in the cognitive activities elicited by vignette-illustrated items; 2) determine whether any differences between performance on vignette-illustrated items and other kinds of items are due to the this form of accommodation’s capacity to address language as a construct-irrelevant factor; 3) identify the set of practical and methodological issues that are critical to properly developing and using vignette-illustrated items; and 4) propose a set of documents and procedures for the systematic and cost-effective design and development of vignette-illustrated items. 

We will test ELL and non-ELL students with items of three types (vignette-illustrated items whose illustrations are designed systematically, vignette-illustrated items whose illustrations are created arbitrarily, and items without illustrations) at two levels of distance to the enacted curriculum (close and distal). Diverse forms of analysis will allow us to determine whether and how ELL and non-ELL students differ on the ways in which they use vignettes to make sense of items, whether the use of vignettes reduces test score differences due to language factors between ELL and non-ELL students, and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.

Intellectual merit. This project will provide information that will help to advance our understanding in two assessment arenas: effective accommodations for ELLs, and item development practices. While illustrations are frequently used in test items, there is not guidance in the assessment development literature on how to approach illustrations. Furthermore, the value of illustrations as a resource for ensuring that ELL students understand what a given item is about and what the item asks them to do has not been systematically investigated. Semiotics, cognitive psychology, and linguistics and socio-cultural theory are brought together to develop systematic procedures for developing illustrations as visual supports in tests. Understanding the role that images play in test taking is relevant to devising more effective ways of testing students. While this project aims to improve testing accommodations practices for ELLs, knowledge gained from it will inform test development practices relevant to all student populations.

Broader impact. We expect outcomes of this project to contribute to enhanced practice in both classroom and large-scale assessment. The push for including ELLs in large-scale testing programs with accountability purposes is not corresponded by effective testing accommodation practices. Many testing accommodations used by national and state assessment programs are not defensible, are not effective, or are improperly implemented. Vignette illustrations have the potential to become a low-cost, easy-to-implement form of testing accommodation for ELLs. Results form this investigation will allow us to identify a set of principles for the proper design and use of vignette illustrations as a form of testing accommodation for ELLs. The project is important not only because it explores the potential of an innovative form of accommodation but because it uses a systematic procedure for designing that form of accommodation.

Learning Assistant Model of Teacher Education in Science and Technology

This project is testing the effectiveness of the 'Learning Assistant Model' for recruiting, preparing, and retaining STEM K-12 teachers by developing a suite of survey instruments that can be used by researchers interested in testing the effectiveness of teacher preparation programs, course transformations, or conceptual or pedagogical knowledge. It focuses on teacher certification programs,K-12 contexts and students' experiences in STEM departments and the role of STEM research faculty in preparing future teachers.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0554616
Funding Period: 
Fri, 09/01/2006 to Tue, 08/31/2010

Toward a Scalable Model of Mathematics Professional Development: A Field Study of Preparing Facilitators to Implement the Problem-solving Cycle

The study includes two and a half years of preparation and support for all the mathematics instructional leaders (ILs) within a large urban school district with a substantial minority student enrollment. These ILs will implement the Problem-Solving Cycle model with the mathematics teachers in their schools. Researchers will analyze the preparation and support that ILs need, the quality of their implementation, and the impact of the PD process on ILs, teachers, and students.

Lead Organization(s): 
Award Number: 
0732212
Funding Period: 
Mon, 10/01/2007 to Thu, 09/30/2010
Full Description: 

The primary goal of the project is to investigate the scalability of the Problem-Solving Cycle (PSC) model of mathematics professional development (PD) and accompanying facilitation materials—that is, whether the PSC can be implemented with integrity by multiple facilitators in multiple settings. In the proposed study we will provide ongoing support to a group of middle school mathematics instructional leaders (ILs) so that they can develop the skills to successfully implement the PSC with the mathematics teachers in their schools. The specific nature of this support is expected to change over the duration of the project, and to gradually decrease as the ILs develop the ability to implement the PSC on their own. Our research will address the following questions:

  1. What preparation is provided to ILs prior to their implementation of the PSC? What support is provided during implementation? How does this support change with successive iterations of the PSC?
  2. How do ILs implement the PSC? How does implementation vary across ILs and over time? What factors account for the variation?
  3. What is the impact of preparation for, and implementation of, the PSC on ILs?
  4. What is the impact of participation in the PSC on middle school mathematics teachers?
  5. What is the impact on the mathematics achievement of students whose teachers participate in the PSC?

 


[1] In this proposal we refer to all school or district personnel who will be trained to facilitate the PSC as “instructional leaders” (ILs) although we recognize that they may have other titles.

A Longitudinal Comparison of the Effects of the Connected Mathematics Program and Other Curricula on Middle School Students' Learning of Algebra

This project compares the effects on algebraic learning when using the Connected Math Program to the effects of using other (non-NSF supported) middle school mathematics curriculum materials at the middle school level. The algebra focus skills/concepts to be assessed are: conceptual understanding and problem solving; algebraic manipulative skills; solution strategies, representations and mathematical justifications.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0454739
Funding Period: 
Tue, 03/01/2005 to Mon, 02/28/2011

Development of K-8 Teachers' Knowledge and the Transition from University Student to Professional

This study examines changes in preservice and inservice K-8 teachers' understandings of science and pedagogy across a reform-based Professional Development Continuum. It researches how teachers' knowledge develops across inquiry learning experiences and how undergraduate learning experiences and the contexts of classroom practice shape new teachers' understandings and practice. The study is expected to inform models of STEM teacher education that account for the interwoven intellectual and personal dimensions of the challenging transition to inquiry-based teaching.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0455781
Funding Period: 
Fri, 07/01/2005 to Wed, 06/30/2010

ASSISTments Meets Inquiry

This project addresses middle school students’ learning of science through the improvement of their inquiry science skills. The main goal is to develop a rigorous, technology-based assessment system for standards-aligned assessment of inquiry skills in six physical science content areas (i.e., Properties of Matter; Elements, Compounds, and Mixtures; Motion of Objects; Forms of Energy; and Heat Energy). Assessments are aligned with the Massachusetts Curricular Framework and National Science Education Standards.

Project Email: 
Partner Organization(s): 
Award Number: 
0733286
Funding Period: 
Sat, 09/01/2007 to Tue, 08/31/2010
Project Evaluator: 
Dr. Carlos Ayala
Full Description: 

PROJECT OVERVIEW: 

The goal of the AMI project (Assistments Meets Inquiry) is to develop a technology-based assessment system for Middle School Physical Science to be aligned with the Massachusetts Curricular Frameworks. We will do this by: 1) leveraging  from the existing  authoring  functionality in the ASSISTments project for Math (developed by Neil Heffernan & colleagues) in order to develop Science Assistments, 2) extending the logging functionality for this system in order to capture students’ fine grained actions with microworlds, 3) evaluating students’ interactions with models using a framework for aggregating students’ actions into domain-general inquiry skills, and 4) extending the existing reporting infrastructure to report students’ inquiry skills to teachers for formative assessment so s/he can determine which skills his/her students are performing poorly on. 

We are working in two urban schools and a small group of suburban/rural schools in the Central MA area. Worcester is the third largest city in Massachusetts, with high levels of children on free- or assisted-lunch programs. The area also is home to children of many different ethnicities, and as such, provides data that are likely to generalize well across the United State

This project has a longitudinal research design and will generate evidence that is causal [experimental, Educational Data Mining].  Original data are being collected using school records, assessments of learning [concept pre- and post-tests for content and inquiry skills], observation [logs], and survey research.  The project is evaluating the use of adaptive scaffolding of inquiry skills within an intelligent tutoring system for science inquiry. No adaptive scaffolding is being used as the comparison condition. The analysis plan for this project involves educational data mining applied to log files of students' log files.

Assessment for Learning Research Scholars: Capacity Building in Mathematics and Science Education

This project seeks to advance knowledge in K-12 STEM education and assessment practices by building capacity for Assessment for Learning, improving assessments and teacher preparation courses, and providing models for pre-service teacher preparation through enhanced teaching modules. Three goals are: (1) faculty from three centers form a learning community, (2) recruit 5 STEM research scholars to conduct research on measurement and evaluation, and (3) expose pre-service teachers to assessment models in their coursework.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
0733590
Funding Period: 
Sat, 09/01/2007 to Tue, 08/31/2010

Pages

Subscribe to Middle School