The goals of this project are to 1) develop methods for analyzing data collected to document the institutional setting of mathematics teaching that are specific to equity and access for all middle school students to high quality mathematics instruction; and 2) develop an instrument for assessing the quality of mathematics instruction that focuses specifically on the extent to which all students are supported to substantially participate in academically rigorous mathematics.
This exploratory research focuses on issues of equity and access to high-quality middle school mathematics instruction in the context of a currently funded project, Designing Learning Organizations for Instructional Improvement in Mathematics (award No. ESI 0554535).
Our overall goal in the Designing Learning Organizations project is to investigate the institutional setting of middle-school mathematics teaching by testing and refining a series of hypotheses about school and district organizational arrangements, material resources, and social relationships (e.g., professional development, teacher networks, shared vision of instruction, relations of accountability and assistance) that might support mathematics teachers' development of high-quality instructional practices at scale.
We are conducting this investigation in four large urban districts that are attempting to achieve a vision of high quality mathematics instruction that is broadly compatible with the National Council of Teachers of Mathematics' (2000) recommendations. As of June 2010, we have completed the third of four rounds of data collection in the Designing Learning Organizations project. Annual rounds of data collection include the following:
1) Audio-recorded interviews and on-line surveys of 120 middle-school teachers (30 in each district)
2) Learning Mathematics for Teaching (LMT) scores of the 120 teachers
3) Video-recordings of two consecutive mathematics lessons in each of the 120 teachers' classrooms
4) Audio-recorded interviews of approximately 80 school and district instructional leaders (approximately 20 in each district)
5) On-line surveys of the principals and coaches in the participating teachers' schools
6) Video-recordings of district-wide and school-based teacher professional development
Issues of equity are central to the districts’ efforts to improve middle school mathematics instruction, particularly the challenge of supporting the substantial participation of groups of low-performing African American students and English Language Learners (ELLs) in academically rigorous mathematics. The overall goal of the SGER project is to develop analytic methods that will enable us to test hypotheses and conjectures regarding equity and access in the Designing Learning Organizations project.The two major activities we proposed for the SGER project include the following:
- development of methods for analyzing data collected to document the institutional setting of mathematics teaching that are specific to equity and access for all middle school students to high quality mathematics instruction.
- development of an instrument for assessing the quality of mathematics instruction that focuses specifically on the extent to which all students are supported to substantially participate in academically rigorous mathematics (particularly traditionally low-performing groups of African American students and ELLs).
Details about the Development of the Equity-Specific Coding Scheme
Based on extant research, our equity-specific conjectures that we are testing in the Designing Learning Organizations project are as follows:
We expect equity in learning opportunities for all students in mathematics at the school level (as measured by improvement in student achievement across sub-populations) to be greater if the following organizational arrangements, material resources, and social relationships exist:
- Detracked instructional program
- Use of a high quality curriculum with groups of under-achieving students (e.g., low-performing African American and ELL students)
- Ongoing professional development specific to supporting low-performing groups of students
- Productive category system for classifying students in relation to mathematics
The last point, “productive category system for classifying students in relation to mathematics,” is heavily informed by the work of Lani Horn (2007), who suggests that the extent to which teachers support the substantial participation of low-performing groups of students in mathematics instruction is related to teachers’ views of mathematics and views of students. Our interviews include a set of questions designed to elicit the categories teachers use to describe students in relation to mathematics.
We finished the development of the equity-specific coding scheme in December 2008. The scheme is divided into five “parent” codes:
- Categories participants used to describe students
- Instructional strategies/differentiation associated with groups of students
- District leaders/instructional leaders’ expectations for teachers regarding differentiation/addressing achievement gaps
- Challenges teachers face, which is meant to capture how many teachers said that a “wide range of abilities” in their classes is a major challenge (we ask teachers to describe the challenges of teaching math in their schools)
- Assumptions about equity and access, which is a code that has emerged to capture general statements/stances that participants take toward equity and access (e.g., “all kids can learn,” “ELLs have an easier time in math than reading”)
- District- and school-based supports for teachers specific to equity and access (e.g., professional development, access to colleagues with equity-specific expertise)
In June 2009, two members of the research team (Kara Jackson and Lynsey Gibbons) completed consensus-coding all 200 Round One interviews conducted with teachers, school leaders, and district leaders. We have begun to analyze the coded data in the following ways to establish a baseline from which to test and refine our conjecture about the role of category systems that teachers use to classify and characterize groups of students in relation to mathematics.
- Analyze the nature, variety and frequency of categories and the characteristics associated with those categories across districts and within schools.
- Analyze the nature and variety of the potential pedagogical moves/actions that teachers say they might take.
- Analyze teachers’ visions of high-quality instruction against their category schemes for classifying students (e.g., we conjecture that teachers with more sophisticated visions of high-quality mathematics instruction will classify and characterize students in more productive ways).
- Analyze instructional leaders’ expectations of how teachers should support all students’ learning.
- Analyze the existence of district- and school-based supports for teachers’ learning that are equity-specific.
We completed an analysis of Round One interview data in one of the four districts; this analysis identified specific school-based supports associated with teachers’ development of ambitious and equitable instructional practices with traditionally low-performing groups of students. Beginning in Spring 2011, we will begin an analysis of change in the quality of learning opportunities that teachers provided (based on our analysis of video data) for traditionally low-performing groups of students from Round Two (January 2009) to Round Three (January 2010) with a focus on organizational arrangements, material resources, and/or social relationships that supported such change.
Details about the Development of Equity-Specific Rubrics to Assess the Quality of Instruction
As part of the Designing Learning Organizations project, we are using the Instructional Quality Assessment (IQA), developed at the University of Pittsburgh (Crosson, Junker, Matsumura, & Resnick, 2003), to code the video-recordings of the participating 120 teachers' classroom lessons in order to document the extent to which the four districts are achieving their agendas for large-scale instructional improvement in mathematics. The IQA is consistent with the ambitious instructional visions of all four districts and focuses on: 1) the cognitive demands of the instructional tasks used in lessons (Stein, Smith, Henningsen, & Silver, 2000), 2) the clarity of expectations for students' learning, and 3) the nature of classroom discourse. However, the IQA does not focus explicitly on issues of equity; it does not attend to the dimensions of classroom practice identified in the research literature as important in ensuring that all students have access to significant mathematical ideas.
The Vanderbilt team, in conjunction with Melissa Boston (Duquesne University, one of the lead developers of the Middle School Mathematics IQA), began intense work on the development of the equity-specific rubrics in Winter 2008-09. We have completed one set of rubrics that focuses on the “task as set up” or “the posing of the task” phase of instruction. Our intent is to provide a valid measure of the relationship between how a task is set up and the extent to which students are able to begin to solve the task (i.e., the extent to which a task is made accessible). (The IQA rubrics do not take into account what happens in the classroom before students begin to work on the task.)
We conjectured that how a task was posed in the classroom would significantly impact the extent to which all students could productively engage in solving the task. Based on viewing our Round One (February 2008) video data collection, we found that in classrooms in which traditionally low-performing students tended to do perform better than expected on state achievement tests, teachers tended to do two things to prepare the students to be able to engage productively in the task. First, when using tasks that ground the mathematics in a problem-solving scenario, the teacher developed students’ familiarity with the cultural suppositions of the problem scenario. Second, regardless of the type of task (e.g., problem-solving scenario, naked number task), the teacher developed students’ situation-specific images of what is to be mathematized in the task (e.g., developed images of the key mathematical ideas, relationships, and/or quantities).
We currently are using the task-as-set-up rubrics, in conjunction with the typical IQA rubrics, to code Round Three of our video-recordings of teachers’ instruction (June-August 2010).
We have also begun work on devising a set of rubrics that attend more carefully to language use in the classroom than the IQA rubrics currently do, with the intent of measuring the quality of instruction that is particular to supporting English language learners to participate in instruction. Lindsey Clare Matsumura (University of Pittsburgh) shared the ELL rubrics the IQA developed in the context of English Language Arts classrooms, and we have begun work on adapting those rubrics to mathematics classrooms. The language rubrics are less developed than the task as set up rubrics. We are intending to spend a significant amount of time drafting the language rubrics June-August 2010, and we will refine and validate the language rubrics in Fall 2010.