This project is exploring how curricula and assessment using dynamic, interactive scientific visualizations of complex phenomena can ensure that all students learn significant science content. Dynamic visualizations provide an alternative pathway for students to understand science concepts, which can be exploited to increase the accessibility of a range of important science concepts. Computer technologies offer unprecedented opportunities to design curricula and assessments using visual technologies and to explore them in research, teaching, and learning.
This project focuses on the challenge of using assessment of relevant STEM content to improve K-12 teaching and learning. CLEAR takes advantage of new technologies and research findings to investigate ways that science assessments can both capture and contribute to cumulative, integrated learning of standards-based concepts in middle school courses. The project will research new forms of assessment that document students' accumulation of knowledge and also serve as learning events.
The CLEAR project takes advantage of new technologies and research findings to investigate ways that science assessments can both capture and contribute to cumulative, integrated learning of standards-based concepts in middle school courses.
Our research investigates how instructional activities can help middle school students develop a cumulative, integrated understanding of energy. Energy is a unifying scientific concept that has been shown to be difficult to learn due to its complexity and abstract nature.
Understanding Science provides an accurate portrayal of the nature of science and tools for teaching associated concepts. This project has at its heart a public re-engagement with science that begins with teacher preparation. To this end, its immediate goals are (1) improve teacher understanding of the nature of the scientific enterprise and (2) provide resources and strategies that encourage and enable K-16 teachers to incorporate and reinforce the nature of science throughout their science teaching.
This project is documenting how middle and secondary school science teachers first develop a perspective on science learning, translate that perspective into their own teaching practice and finally make explicit links to their colleagues. The research is investigating the impact of professional development based on model-based reasoning, supported by Lesson Study and an apprentice-like program in teacher leadership.
This project studies teaching practices in a year-long high school algebra course that integrates hand-held and other electronic devices. Of particular interest is how these technologies can support learners' capacity to efficiently and effectively draw on the distributed intelligences that technical and social networks make available. The investigation focuses on collaborative learning tasks centered on collective mathematical objects, such as functions, expressions, and coordinates that participants in a group must jointly manipulate through networked computers.
This project uses new psychometric techniques to create a technological tool that could evaluate how well students in the 4th-8th mathematics and science classrooms respond to complex performance tasks. The purpose of this tool is to improve the instruction of teachers in mathematics and science. It will produce real-time individualized diagnoses of instructional needs to help teachers plan instruction that specifically addresses the learning needs of each student in that class.
This project addresses the Measurement goal under the Mathematics and Science Education research program. Specifically, we propose developing and refining an assessment development, delivery, scoring, and report-generating system in the area of mathematics, centered on statistics and modeling. We have been engaged with colleagues at Vanderbilt University in designing a formative assessment system to support (and help evaluate) their innovative curriculum in this area: the Assessing Data Modeling and Statistical Reasoning (ADM) system developed by Rich Lehrer and his colleagues (Lehrer & Schauble, 2007). This assessment system has been created using the principles of the BEAR Assessment System (BAS; Wilson, 2005), and it and the curriculum it supports is currently being used in several states (WI, AK, TN), and is being adopted into a broader curriculum that is widely used. The aim of the current project is: (a) to refine a set of software programs that the Berkeley Evaluation and Assessment Research (BEAR) Center has been developing over the last 10 years that support the development, calibration, use and training for the assessment system, and to develop software interconnections among those programs to allow them to operate seamlessly for users whose roles range from assessment developers to teachers to school administrators, to those who provide professional development for teachers; (b) as a first full trial of that software, to embed the existing ADM materials in the software, construct computer-deliverable and computer-scorable task equivalents of the current item bank, and develop new computerized reports and support materials for teachers; and (c) to investigate the usefulness of this new software in the context of the ADM curriculum.
The positive effects of innovative assessments are widely acknowledged (Black & Wiliam, 1998), and we are happy that the BAS is seen as one such innovation. But we are strongly concerned that the good effects that one can find from early-adopters of such innovations will not be sustained unless the considerable burden of teacher scoring of their students’ formative assessments is lightened. We believe that it is essential that teachers become experts in interpreting their student’s responses to assessments. But, equally, we see that it is wise to then relieve them of the burden of continual scoring of large amounts of student work. Hence, the strategy we have adopted is to involve teachers early on in a deep program of professional development that will include close work with curriculum materials, assessments and student responses to assessments (preferably including a large proportion of work from their own students). However, once teachers have shown their mastery of the role of scorer and interpreter of such student products, we then provide the teacher with computerized assessments that will deliver and score equivalent assessments for their students, and generate rich interpretational materials to help them with diagnosis and planning. We expect that teachers will still be called upon to evaluate unusual student responses, and also will need to carry out occasional hand-scoring to keep up their mastery and to adapt to innovations in the curriculum.
This project focuses on the assessed impact of a teacher professional development (TPD) program around the applied sciences. Specifically, researchers seek to examine the measurable impact of an established teacher professional development program currently offered through the UC Davis Edward Teller Education Center. The Center delivers teacher training and curricula and draws upon an instructor cohort that pairs a regional master teacher and relevant LLNL scientists in curriculum development and delivery.
This project covers participants' costs to attend a national conference series focusing upon supporting incipient science education research projects. A primary objective is to provide a venue in which researchers can describe their lines of inquiry and to then receive guidance and input about refining those ambitions. The other primary objective is to promote an innovative conference design in which a structured presentation format serves as an incubator for scholarly work.
This project investigates how vignette illustrations minimize the impact of limited English proficiency on student performance in science tests. Different analyses will determine whether and how ELL and non-ELL students differ significantly on the ways they use vignettes to make sense of items; whether the use of vignettes reduces test-score differences due to language factors between ELL and non-ELL students; and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.
This exploratory project within the Contextual Strand (Challenge a) addresses validity in the assessment of science and mathematics for English language learners (ELLs), and the urgent need for effective testing accommodations for ELLs. Motivation for this investigation originated from a previous, NSF-funded project on the testing of ELLs. We observed that items which were accompanied by illustrations tended to be responded correctly by a higher percentage of students than items without illustrations. We will investigate the factors that are relevant to designing and using a new form of accommodation in the assessment of science and mathematics for ELLs—vignette illustrations.
This three-year project will be guided by four research questions: What principles underlie the effective design of science and mathematics test items with illustrations in ways that minimize limited English proficiency as a factor that prevents ELLs from understanding the items? Is the presence of an illustration a moderator in students’ understanding test items? If so, Is the effect due to the simple presence of a graphical component or due to characteristics of the illustrations that are created based on principled design? Does the presence of an illustration have a different effect on the performance of ELLs and the performance of non-ELL students?
We expect to be able to: 1) identify the role of illustrations in the cognitive activities elicited by vignette-illustrated items; 2) determine whether any differences between performance on vignette-illustrated items and other kinds of items are due to the this form of accommodation’s capacity to address language as a construct-irrelevant factor; 3) identify the set of practical and methodological issues that are critical to properly developing and using vignette-illustrated items; and 4) propose a set of documents and procedures for the systematic and cost-effective design and development of vignette-illustrated items.
We will test ELL and non-ELL students with items of three types (vignette-illustrated items whose illustrations are designed systematically, vignette-illustrated items whose illustrations are created arbitrarily, and items without illustrations) at two levels of distance to the enacted curriculum (close and distal). Diverse forms of analysis will allow us to determine whether and how ELL and non-ELL students differ on the ways in which they use vignettes to make sense of items, whether the use of vignettes reduces test score differences due to language factors between ELL and non-ELL students, and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.
Intellectual merit. This project will provide information that will help to advance our understanding in two assessment arenas: effective accommodations for ELLs, and item development practices. While illustrations are frequently used in test items, there is not guidance in the assessment development literature on how to approach illustrations. Furthermore, the value of illustrations as a resource for ensuring that ELL students understand what a given item is about and what the item asks them to do has not been systematically investigated. Semiotics, cognitive psychology, and linguistics and socio-cultural theory are brought together to develop systematic procedures for developing illustrations as visual supports in tests. Understanding the role that images play in test taking is relevant to devising more effective ways of testing students. While this project aims to improve testing accommodations practices for ELLs, knowledge gained from it will inform test development practices relevant to all student populations.
Broader impact. We expect outcomes of this project to contribute to enhanced practice in both classroom and large-scale assessment. The push for including ELLs in large-scale testing programs with accountability purposes is not corresponded by effective testing accommodation practices. Many testing accommodations used by national and state assessment programs are not defensible, are not effective, or are improperly implemented. Vignette illustrations have the potential to become a low-cost, easy-to-implement form of testing accommodation for ELLs. Results form this investigation will allow us to identify a set of principles for the proper design and use of vignette illustrations as a form of testing accommodation for ELLs. The project is important not only because it explores the potential of an innovative form of accommodation but because it uses a systematic procedure for designing that form of accommodation.
This project is testing the effectiveness of the 'Learning Assistant Model' for recruiting, preparing, and retaining STEM K-12 teachers by developing a suite of survey instruments that can be used by researchers interested in testing the effectiveness of teacher preparation programs, course transformations, or conceptual or pedagogical knowledge. It focuses on teacher certification programs,K-12 contexts and students' experiences in STEM departments and the role of STEM research faculty in preparing future teachers.