This project seeks to advance knowledge in K-12 STEM education and assessment practices by building capacity for Assessment for Learning, improving assessments and teacher preparation courses, and providing models for pre-service teacher preparation through enhanced teaching modules. Three goals are: (1) faculty from three centers form a learning community, (2) recruit 5 STEM research scholars to conduct research on measurement and evaluation, and (3) expose pre-service teachers to assessment models in their coursework.
This project is creating five video-case modules for use in professional development of middle school mathematics teachers. The materials are designed to develop teachers' understanding of mathematics knowledge for teaching similarity. In total, 18-24 video cases will be produced, which, taken together, form the curriculum of a 45- to 60-hour professional development course.
The primary purpose of this international conference was for participants in the US to exchange views and discuss the latest research findings on (primary) science assessment. The conference focused on research around building assessment systems that help teachers diagnose student learning in the classroom but also link meaningfully to large-scale accountability systems (in districts or national levels). The project resulted in a report, proceedings, journal publications.
This project studies mathematics professional development leaders' understandings and practices associated with developing mathematically rich learning environments. It investigates this issue by considering: How can leaders cultivate professional development environments in which teachers have a greater opportunity to grapple with and deeply understand mathematics? The project studies how explicit attention to the cultivation of sociomathematical norms influences leaders' understanding of the process of creating mathematically rich environments and the impacts on their practices.
Our research and development work focuses on one aspect of mathematics professional development, when teachers are engaged in solving, discussing, and sharing mathematical work. Although mathematics professional development may include other activities, we specifically focus on how leaders learn to attend to doing mathematics with teachers because it is a primary time during PD that teachers may be developing deeper understandings of mathematics. To support their learning about cultivating rich teacher learning environments, leaders explored two frameworks: sociomathematical norms (norms for mathematical reasoning) and a set of practices for orchestrating productive mathematical discussions. The staff of RMLL created and facilitated seminars as learning opportunities for leaders, studied what and how leaders learned about facilitation, and investigated how leaders facilitated PD in their schools and districts.
As our research project has evolved, we have revised our frameworks for supporting leader development to include a focus on identifying the purposes for doing mathematics with teachers. We have used Deborah Ball and her colleagues' work at the University of Michigan to draw a distinction between common content knowledge that teachers hold in common with other professional using mathematics and specialized content knowledge that teachers need to know because of their unique role in We engage in mathematics with teachers in professional development to help them develop not just common content knowledge but specialized knowledge as well. To develop specialized mathematical knowledge, teachers need to engage in explanations that make taken-for-granted ideas in mathematics explicit. Norms for explanation and representational use are vital. These norms are fostered through the orchestration of discussions. In redesigning seminars according to these ideas, we aim to have leaders select and design tasks that engage teachers more comprehensively with the mathematical knowledge they need to teach. Leaders need to know how to specify purposes for doing mathematics in ways that develop teachers’ SCK and identify tasks and discussion prompts that immerse teachers in SCK. They need to know how to pursue this purpose when orchestrating discussions and support the development of sociomathematical norms in ways that unpack teachers’ highly symbolic or incomplete reasoning. In short, we augmented our initial emphasis on sociomathematical norms with this new emphasis on SCK. supporting learners in the classroom.
We are completing analyses of the experiences of leaders in our revised seminars to understand what they gained from our revised frameworks in planning for and enacting professional development.
This project is conducting an empirical analysis of NAEP assessment items in science to determine whether evidence supports the hypothesis that standardized tests capture only a limited amount of student knowledge because of their cultural background. The investigator will create a model of test design more likely to extract student knowledge from students of varied cultures by expanding items’ content. The study will examine the experience of American Indian groups, Alaska Natives, and Pacific Islanders.
This project has pioneered simulation-based assessments of model-based science learning and inquiry practices for middle school physical and life science systems. The assessment suites include curriculum-embedded, formative assessments that provide immediate, individualized feedback and graduated coaching with supporting reflection activities as well as summative end-of-unit benchmark assessments. The project has documented the instructional benefits, feasibility, utility, and technical quality of the assessments with over 7,000 students and 80 teachers in four states.
Math Pathways & Pitfalls lessons for students boost mathematics achievement for diverse students, including English Learners, English Proficient students, and Latino students. This project develops modules that increase teachers’ capacity to employ the effective and equitable principles of practice embodied by Math Pathways & Pitfalls and apply these practices to any mathematics lesson. This four-year project develops, field tests, and evaluates 10 online professional development modules.
Researchers and developers at WestEd are developing, field-testing, and evaluating ten online professional development modules anchored in research-based teaching principles and achievement-boosting mathematics materials. The modules provide interactive learning opportunities featuring real classroom video demonstrations, simulations, and scaffolded implementation. The professional development module development builds on the Math Pathways and Pitfalls instructional modules for elementary and middle school students developed with NSF support. The professional development provided through the use of these modules is web-based (rather than face-to-face), is provided in chunks during the school year and immediately applied in the classroom (rather than summer professional development and school year application), and explicitly models ways to apply key teaching principles to regular mathematics lessons (rather than expecting teachers to extract and apply principles spontaneously).
The project studies the impact of the modules on teaching practice with an experimental design that involves 20 treatment teachers and 20 control teachers. Data are gathered from teacher questionnaires, classroom observations, and post-observation interviews.
MIST is a five-year study of four large, urban districts implementing ambitious mathematics reform initiatives in the middle grades. The study uses a mixed-methods research design to investigate how changes in the school and district settings in which mathematics teachers work influence their instructional practices, students' learning opportunities, and student achievement.
The research base on supporting mathematics teachers' development of ambitious instructional practices at scale is thin in both mathematics education and in policy and leadership.
Funding agencies including NSF have invested heavily in ambitious agendas for teacher professional development in mathematics. Prior large-scale improvement efforts that have attempted to penetrate the instructional core of the classroom have rarely produced lasting changes in teachers’ instructional practices (Elmore, 2004; Gamoran et al., 2003).
This project is designed to examine the institutional settings in which the classroom is situated (i.e. the district and school environment) with the goal of supporting teacher professional development and causing lasting change in instruction at the classroom level.
Prior NSF-funded initiatives made an important contribution by focusing on a singleaspect of the institutional settings in which mathematics teachers develop and revise their instructional practices: either 1) Principals’ knowledge of mathematics and their beliefs about mathematics teaching and learning; 2) The content, pedagogical, and diagnostic knowledge necessary for leaders to assist mathematics teachers effectively; or 3) Districts’ use of instructional guidance tools such as pacing guidelines and alignment charts.
Our primary goal in this project is to investigate, test, and refine a set of conjectures regarding the support structures needed to enhance the impact of professional development on mathematics teachers’ instructional practices and thus student achievement. In addressing this goal, we will take a comprehensive view of the institutional setting of mathematics teaching rather than focusing on a single aspect.
The support structures on which we will focus include 1) Teacher learning communities and informal networks, 2) Shared vision for mathematics instruction (as indicated by use of a common language for talking about mathematics instruction, presence of brokers who can bridge perspectives, and compatible interpretations of key boundary objects such as instructional materials and state standards and assessments), 3) Distribution of instructional leadership across formal and informal leaders, 4) Reciprocal accountability between teachers and instructional leaders (as indicated by alignment of assistance and accountability and access to key resources such as coherent instructional guidance instruments), and 5) Depth of instructional leaders’ understanding of mathematics, the instructional program, and the challenges of using it effectively.
We will investigate our conjectures by employing a mixed methods design that involves both a formal hypothesis-testing component and design research component. We will work in four urban school districts over four years. The data we will collect or document includes: 1) The institutional setting of teaching (i.e., the above support structures), 2) Teachers’ instructional practices and content knowledge for teaching, 3) The professional development activities in which teachers participate, 4) Formal and informal leaders’ instructional leadership practices, and 5) Student achievement.
The overall product of the two components will be a framework for guiding, monitoring, and assessing school and district-wide institutional improvement in mathematics. This Institutional Improvement Framework will identify the support structures that our findings document are important, explain why they are important and under what conditions, clarify how they are interdependent, and illustrate how their development can be accomplished.
This project is focusing on the redesign of popular commercial video games to support students’ understanding of Newtonian mechanics. In support of this goal, SURGE develops and implements design principles for game-based learning environments, integrating research on conceptual change, cognitive processing-based design, and socio-cognitive scripting. These enhanced games bridge the gap between student learning in non-formal game environments and the formalized knowledge structures learned in school by leveraging and integrating the strengths of each.
The goals of this project are to 1) develop methods for analyzing data collected to document the institutional setting of mathematics teaching that are specific to equity and access for all middle school students to high quality mathematics instruction; and 2) develop an instrument for assessing the quality of mathematics instruction that focuses specifically on the extent to which all students are supported to substantially participate in academically rigorous mathematics.
This exploratory research focuses on issues of equity and access to high-quality middle school mathematics instruction in the context of a currently funded project, Designing Learning Organizations for Instructional Improvement in Mathematics (award No. ESI 0554535).
Our overall goal in the Designing Learning Organizations project is to investigate the institutional setting of middle-school mathematics teaching by testing and refining a series of hypotheses about school and district organizational arrangements, material resources, and social relationships (e.g., professional development, teacher networks, shared vision of instruction, relations of accountability and assistance) that might support mathematics teachers' development of high-quality instructional practices at scale.
We are conducting this investigation in four large urban districts that are attempting to achieve a vision of high quality mathematics instruction that is broadly compatible with the National Council of Teachers of Mathematics' (2000) recommendations. As of June 2010, we have completed the third of four rounds of data collection in the Designing Learning Organizations project. Annual rounds of data collection include the following:
1) Audio-recorded interviews and on-line surveys of 120 middle-school teachers (30 in each district)
2) Learning Mathematics for Teaching (LMT) scores of the 120 teachers
3) Video-recordings of two consecutive mathematics lessons in each of the 120 teachers' classrooms
4) Audio-recorded interviews of approximately 80 school and district instructional leaders (approximately 20 in each district)
5) On-line surveys of the principals and coaches in the participating teachers' schools
6) Video-recordings of district-wide and school-based teacher professional development
Issues of equity are central to the districts’ efforts to improve middle school mathematics instruction, particularly the challenge of supporting the substantial participation of groups of low-performing African American students and English Language Learners (ELLs) in academically rigorous mathematics. The overall goal of the SGER project is to develop analytic methods that will enable us to test hypotheses and conjectures regarding equity and access in the Designing Learning Organizations project.The two major activities we proposed for the SGER project include the following:
- development of methods for analyzing data collected to document the institutional setting of mathematics teaching that are specific to equity and access for all middle school students to high quality mathematics instruction.
- development of an instrument for assessing the quality of mathematics instruction that focuses specifically on the extent to which all students are supported to substantially participate in academically rigorous mathematics (particularly traditionally low-performing groups of African American students and ELLs).
Details about the Development of the Equity-Specific Coding Scheme
Based on extant research, our equity-specific conjectures that we are testing in the Designing Learning Organizations project are as follows:
We expect equity in learning opportunities for all students in mathematics at the school level (as measured by improvement in student achievement across sub-populations) to be greater if the following organizational arrangements, material resources, and social relationships exist:
- Detracked instructional program
- Use of a high quality curriculum with groups of under-achieving students (e.g., low-performing African American and ELL students)
- Ongoing professional development specific to supporting low-performing groups of students
- Productive category system for classifying students in relation to mathematics
The last point, “productive category system for classifying students in relation to mathematics,” is heavily informed by the work of Lani Horn (2007), who suggests that the extent to which teachers support the substantial participation of low-performing groups of students in mathematics instruction is related to teachers’ views of mathematics and views of students. Our interviews include a set of questions designed to elicit the categories teachers use to describe students in relation to mathematics.
We finished the development of the equity-specific coding scheme in December 2008. The scheme is divided into five “parent” codes:
- Categories participants used to describe students
- Instructional strategies/differentiation associated with groups of students
- District leaders/instructional leaders’ expectations for teachers regarding differentiation/addressing achievement gaps
- Challenges teachers face, which is meant to capture how many teachers said that a “wide range of abilities” in their classes is a major challenge (we ask teachers to describe the challenges of teaching math in their schools)
- Assumptions about equity and access, which is a code that has emerged to capture general statements/stances that participants take toward equity and access (e.g., “all kids can learn,” “ELLs have an easier time in math than reading”)
- District- and school-based supports for teachers specific to equity and access (e.g., professional development, access to colleagues with equity-specific expertise)
In June 2009, two members of the research team (Kara Jackson and Lynsey Gibbons) completed consensus-coding all 200 Round One interviews conducted with teachers, school leaders, and district leaders. We have begun to analyze the coded data in the following ways to establish a baseline from which to test and refine our conjecture about the role of category systems that teachers use to classify and characterize groups of students in relation to mathematics.
- Analyze the nature, variety and frequency of categories and the characteristics associated with those categories across districts and within schools.
- Analyze the nature and variety of the potential pedagogical moves/actions that teachers say they might take.
- Analyze teachers’ visions of high-quality instruction against their category schemes for classifying students (e.g., we conjecture that teachers with more sophisticated visions of high-quality mathematics instruction will classify and characterize students in more productive ways).
- Analyze instructional leaders’ expectations of how teachers should support all students’ learning.
- Analyze the existence of district- and school-based supports for teachers’ learning that are equity-specific.
We completed an analysis of Round One interview data in one of the four districts; this analysis identified specific school-based supports associated with teachers’ development of ambitious and equitable instructional practices with traditionally low-performing groups of students. Beginning in Spring 2011, we will begin an analysis of change in the quality of learning opportunities that teachers provided (based on our analysis of video data) for traditionally low-performing groups of students from Round Two (January 2009) to Round Three (January 2010) with a focus on organizational arrangements, material resources, and/or social relationships that supported such change.
Details about the Development of Equity-Specific Rubrics to Assess the Quality of Instruction
As part of the Designing Learning Organizations project, we are using the Instructional Quality Assessment (IQA), developed at the University of Pittsburgh (Crosson, Junker, Matsumura, & Resnick, 2003), to code the video-recordings of the participating 120 teachers' classroom lessons in order to document the extent to which the four districts are achieving their agendas for large-scale instructional improvement in mathematics. The IQA is consistent with the ambitious instructional visions of all four districts and focuses on: 1) the cognitive demands of the instructional tasks used in lessons (Stein, Smith, Henningsen, & Silver, 2000), 2) the clarity of expectations for students' learning, and 3) the nature of classroom discourse. However, the IQA does not focus explicitly on issues of equity; it does not attend to the dimensions of classroom practice identified in the research literature as important in ensuring that all students have access to significant mathematical ideas.
The Vanderbilt team, in conjunction with Melissa Boston (Duquesne University, one of the lead developers of the Middle School Mathematics IQA), began intense work on the development of the equity-specific rubrics in Winter 2008-09. We have completed one set of rubrics that focuses on the “task as set up” or “the posing of the task” phase of instruction. Our intent is to provide a valid measure of the relationship between how a task is set up and the extent to which students are able to begin to solve the task (i.e., the extent to which a task is made accessible). (The IQA rubrics do not take into account what happens in the classroom before students begin to work on the task.)
We conjectured that how a task was posed in the classroom would significantly impact the extent to which all students could productively engage in solving the task. Based on viewing our Round One (February 2008) video data collection, we found that in classrooms in which traditionally low-performing students tended to do perform better than expected on state achievement tests, teachers tended to do two things to prepare the students to be able to engage productively in the task. First, when using tasks that ground the mathematics in a problem-solving scenario, the teacher developed students’ familiarity with the cultural suppositions of the problem scenario. Second, regardless of the type of task (e.g., problem-solving scenario, naked number task), the teacher developed students’ situation-specific images of what is to be mathematized in the task (e.g., developed images of the key mathematical ideas, relationships, and/or quantities).
We currently are using the task-as-set-up rubrics, in conjunction with the typical IQA rubrics, to code Round Three of our video-recordings of teachers’ instruction (June-August 2010).
We have also begun work on devising a set of rubrics that attend more carefully to language use in the classroom than the IQA rubrics currently do, with the intent of measuring the quality of instruction that is particular to supporting English language learners to participate in instruction. Lindsey Clare Matsumura (University of Pittsburgh) shared the ELL rubrics the IQA developed in the context of English Language Arts classrooms, and we have begun work on adapting those rubrics to mathematics classrooms. The language rubrics are less developed than the task as set up rubrics. We are intending to spend a significant amount of time drafting the language rubrics June-August 2010, and we will refine and validate the language rubrics in Fall 2010.