This project builds on the study of the Ongoing Assessment Project's (OGAP) math assessment intervention on elementary teachers and students and combines the intervention with research-based understandings of systemic reform. This project will produce concrete tools, routines, and practices that can be applied to strengthen programs' implementation by ensuring the strategic support of school and district leaders.
Districts have long struggled to implement instructional programming in ways that meaningfully and sustainably impact teaching and learning. Systemic education reform is based on the hypothesis that prevailing patterns of incoherence and misalignment in an educational system can send mixed messages to local implementers as they try to respond to various cues and incentives in the environment. Systemic reform seeks to bring alignment to education systems in multiple ways, including consistency across instructional philosophies, alignment across grade levels, and vertical coherence from district to schools to classrooms. This project builds on the Consortium for Policy Research in Education's (CPRE) ongoing, NSF-funded experimental study of the impacts of the Ongoing Assessment Project's (OGAP) math assessment intervention on elementary teachers and students in Philadelphia-area schools. The project will combine the OGAP math intervention with research-based understandings of systemic reform. OGAP is based upon established theory and research demonstrating the impact of teachers' use of ongoing short- and medium-cycle formative assessment on student learning. It combines these understandings with recent research on learning trajectories within mathematics content domains. By bringing to bear the strengths of all three of these areas of research - formative assessment, learning trajectories, and systemic reform - the project promises a significant contribution to the knowledge base about the application of math learning research to classroom instruction on a large scale. This project will produce concrete tools, routines, and practices that can be applied to strengthen programs' implementation by ensuring the strategic support of school and district leaders. This project is funded by the Discovery Research PreK-12 (DRK-12) and EHR Core Research (ECR) Programs. The DRK-12 program supports research and development on STEM education innovations and approaches to teaching, learning, and assessment. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field.
CPRE and the School District of Philadelphia (SDP) will establish a research-practice partnership focused on developing, implementing, refining, and testing a systemic support model to strengthen implementation of the OGAP math intervention in elementary schools. CPRE's current experimental study of OGAP's impacts reveals, preliminarily, statistically significant positive effects on teacher knowledge and student learning. As a result, SDP has decided to expand OGAP into an additional 60 schools in 2016-17. However, the current OGAP study has also revealed weak implementation stemming from a lack of consistent leadership support for the intervention. The project will address this implementation challenge by developing, refining, supporting, and documenting a systemic support component that will accompany OGAP's classroom-level implementation. The systemic supports will be developed by a research-practice partnership between CPRE; SDP; OGAP; the Graduate School of Education at the University of Pennsylvania (PennGSE); and the Philadelphia Education Research Consortium (PERC). The team will use principles of design-based implementation research to iteratively refine and improve the systemic support model. Along with the design and development of the systemic support model, the project will conduct a mixed-methods study of its impacts and roll-out. A three-armed quasi-experimental study will examine the differential impacts of OGAP, with and without systemic supports, and business-as-usual math programming on teacher and student outcomes. A mixed-methods study will examine teacher and administrator experiences in both treatment groups, and will provide feedback to inform the iterative development of the systemic support model.
CAREER: A Study of Factors that Affect Middle School Levels of Readiness for Implementing STEM Programs
This project will investigate whether six urban middle schools are implementing highly effective science, technology, engineering and mathematics (STEM) programs based on factors identified through relevant research and national reports on what constitutes exemplary practices in 21st century-focused schools.
This is a Faculty Early Career Development Program (CAREER) proposal responsive to Program Solicitation NSF 15-555. The CAREER program is a National Science Foundation-wide activity that offers the most prestigious awards in support of junior faculty who exemplify the role of teacher-scholars through outstanding research, excellent education and the integration of education and research. This project will investigate whether six urban middle schools are implementing highly effective science, technology, engineering and mathematics (STEM) programs based on factors identified through relevant research and national reports on what constitutes exemplary practices in 21st century-focused schools. The project will make this determination through the use of a STEM level of readiness rubric developed through a previous award that will be further revised through this study. The rubric will document the participating schools' level of readiness at the principal, teacher, and student levels using 15 criteria that include a combination of essential supports, core elements, attributes, and characteristics about STEM through: (1) school leadership as the driver of change in education; (2) professional capacity among teachers and staff in all academic areas; (3) student-centered learning climate reflective of high-quality teaching and learning practices; and (4) investment of resources (e.g. staffing, time, space, materials and supplies, partnerships) that support exemplary school-based programs.
The project will use surveys, focus groups, and face-to-face interviews to collect data from 18 principals; classroom observations and a survey to collect data from 380 teachers, and a survey to collect data from 3700 students. These data collections, augmented by other intermittent research activities, will provide insights about extant programs in participating schools regarding effective school leadership, state-of-the art teaching and learning practices, and the impact on students' interest, motivation, and self-efficacy about STEM education. The primary outcome from this project will be a field-tested jointly refined STEM level of readiness rubric based on input from principals, teachers, and students with guidance from the project's advisory board and the Center for Research in Educational Policy at the University of Memphis. The rubric will be instrumental in informing district-level education stakeholders and university-partner decision-makers' choices about where and when to invest resources to further support the development of higher quality STEM programs and schools. It will also be useful in identifying ways to improve students' overall perceptions about future courses of study and careers and the development of professional development modules for teacher training. Beyond these key school district-level outcomes, results will be used to enhance teacher preparation efforts through further refinement of methods courses and the STEM Teacher Leadership Certificate Program at the University.
This project investigates the variation in teachers' practice of lesson study to identify effective and scalable design features of lesson study associated with student mathematics achievement growth in Florida. Lesson study is a teacher professional development model in which a group of teachers works collaboratively to plan a lesson, observe the lesson in a classroom with students, and analyze and discuss the student work and understanding in response to the lesson.
This project investigates the variation in teachers' practice of lesson study to identify effective and scalable design features of lesson study associated with student mathematics achievement growth in Florida. Lesson study is a teacher professional development model in which a group of teachers works collaboratively to plan a lesson, observe the lesson in a classroom with students, and analyze and discuss the student work and understanding in response to the lesson. Florida is the first state to promote lesson study as a statewide professional development model for implementing the Common Core State Standards for Mathematics and improving instruction and student achievement. The original lesson study model imported from Japan poses a challenge for implementation and scalability in the United States, and there is emerging evidence that modifications have been made to make it feasible within the constraints of teachers' work schedules and school structures. Thus, there is an urgent need to investigate the variation in lesson study practice and how modified design features of mathematics lesson study are associated with improvement of student mathematics achievement. The research team will conduct a statewide survey of approximately 1,000 teachers in grades 3-8 who are practicing mathematics lesson study during the 2015-2016 academic year. They will examine variations in four design features of lesson study (structure, facilitator, knowledge resources for lesson planning, and research lesson and discussion) and their associated organizational supports. They will examine the relationships between these design features and the original lesson study model, teacher learning, and students' mathematics achievement growth.
This project is designed to advance the scholarship and practice of lesson study by: (1) identifying an effective and scalable model of mathematics lesson study with specific design features that are associated with positive teacher learning experience and improved student mathematics achievement; (2) advancing practical knowledge on how this effective and scalable model of mathematics lesson study can be practiced, based on in-depth case studies of lesson study groups; and (3) contributing to teacher learning principles that can be applied to various professional development programs in mathematics. The project will disseminate evidence regarding the characteristics of an effective and scalable mathematics lesson study model to state and district-level facilitators across the country. The project will also develop a Florida Lesson Study Network (FLSN) to share resources and facilitate communications regarding lesson study practice.
This project will convene a panel of experts in government, industry and academia to raise and discuss emerging concerns for human subjects' protections in the digital age. This project will support scholarly discussion on human subjects' protections in the digital age with implications for funding agencies, schools, and those who work with human subjects in a variety of environments.
This project will convene a panel of experts in government, industry and academia to raise and discuss emerging concerns for human subjects' protections in the digital age. Learners taking part in formal education, informal education, and out-of-school settings are subject to a ubiquitous tracking of their activities: locally, using the internet of things (e.g., smart phones, smart sensors and other cyberphysical devices), and globally, via the internet. This tracking may include data tracked passively (e.g., online purchases) or data made available on social media websites by the learners themselves. In addition, the use of the longitudinal data collected by local educational agencies for research is an increasingly political concern. Decisions about the use of these data by university researchers and scholars are typically made by Institutional Review Board (IRB) offices. New guidelines on IRB practices are being considered by a number of bodies, including the National Research Council, which issued a report in early 2014.
This project will support scholarly discussion on human subjects' protections in the digital age with implications for funding agencies, schools, and those who work with human subjects in a variety of environments. The issues discussed are of national import, including, but not limited to FERPA privacy concerns. To the extent that US researchers work with data from other countries (e.g., via massive open online courses or MOOCs), the impact of the reports that will be produced as part of this conference for education research is potentially global.
Testing a Professional Development Model for High School Science Reform and the Relationship of Key Variables to Student Achievement
This project tests the efficacy of an intensive, three year professional development program, the BSCS National Academy for Curriculum Leadership (NACL) on student science achievement in the state of Washington. The goal of the NACL is to develop the capacity of district-based secondary science leadership teams to sustain the implementation of research-based science instructional materials that promote improvement in teaching and learning.
This project conducted by the Biological Sciences Curriculum Study(BSCS) tests the efficacy of an intensive, three year professional development program, the BSCS National Academy for Curriculum Leadership (NACL) on student science achievement in the state of Washington. The goal of the NACL is to develop the capacity of district-based secondary science leadership teams to sustain the implementation of research-based science instructional materials that promote improvement in teaching and learning. This study examines the influence of the program on student achievement after the schools and districts have had sufficient time for the effects to take hold.
The project uses existing data gathered from two cohorts of Washington-based NACL teams and archived student achievement data from Washington State?s Office of the Superintendent of Public Instruction (OSPI). Additional data are collected NACL-participating districts and personnel through surveys and interviews. The project compares student achievement between the 27 districts that participated in the NACL, and a minimum of 27 carefully matched, non-NACL districts using propensity-scoring methodology. Districts have experienced different dosages of the NACL, and the project examines the differential effects of being involved in the NACL over time.
This research study provides an opportunity for multiple stakeholders including NSF, other corporate foundations that have funded the development and implementation of the NACL, BSCS, and participating school districts to determine the extent to which professional development promotes the improvement student science achievement results. The broader impact of the research is testing the extent to which basic elements of teacher professional development models correlate with student achievement and to do so in a way that could be replicated by others in similar contexts. The proposed work would inform educators about the research-based approaches to professional development that has evidence of efficacy. Moreover, by determining the time-scales by which professional development programs might be shown to influence student achievement, the findings provide new information to policymakers and researchers regarding the amount of time that could be required to see a positive impact from new educational policies and programs.
Cross-National Comparison of School and District Supports for High-Quality Mathematics Instruction in the US and China
This RAPID project is a cross-national comparative study of U.S. and Chinese instructional support systems, building from earlier data about mathematics teaching and learning in large urban school districts of both the United States and the People's Republic of China. The study uses quantitative methods to compare and contrast the effectiveness of supports (e.g., professional development, teacher networks, school leadership) in improving teachers' instructional practices and student achievement using comparable instrumentation.
Since the publication of the result that students from Shanghai, China, outperformed students from all other participating countries on the 2009 Programme for International Student Assessment (PISA) in mathematics, researchers have sought to understand why Chinese mathematics education appears to be both more successful at boosting student learning and more equitably distributed. This RAPID project is a cross-national comparative study of U.S. and Chinese instructional support systems, building from earlier data about mathematics teaching and learning in large urban school districts of both the United States and the People's Republic of China. The work is being conducted by researchers from Vanderbilt University, Virginia Polytechnic Institute and State University and Beijing Normal University. The study uses quantitative methods to compare and contrast the effectiveness of supports (e.g., professional development, teacher networks, school leadership) in improving teachers' instructional practices and student achievement using comparable instrumentation.
The study contributes to research and policy in several ways. First, it is helping to identify supports that have been particularly effective in improving mathematics teaching and learning in China. This should inform current theories about how to best support mathematics education in the United States. Second, the cross-nationally validated instruments used to collect the data can be used by other researchers investigating curricular reform implementation cross-nationally. The findings of this study are especially relevant to district leaders as they develop support and accountability systems to effectively implement the content and practice standards of the Common Core State Standards for Mathematics.
This award is co-funded by NSF's International Science and Engineering Section, Office of International and Integrative Activities.
This is a quasi-experimental study of the effects of attending an inclusive STEM high school in three key geographic regions and comparing outcomes for students in these schools with those of their counterparts attending other types of schools in the same states. The study's focus is on the extent to which inclusive STEM high schools contribute to improved academic outcomes, interests in STEM careers, and expectations for post secondary study.
Researchers from SRI and George Washington University are studying the effectiveness of inclusive STEM high schools in three key geographic regions including Texas, North Carolina and Ohio. STEM schools continue to be an important policy area and test bed for one indication of what STEM education can accomplish under the most optimal conditions in which STEM is the focus of students' learning experiences. The President has called for the creation of an additional 1,000 STEM schools with relatively little evidence about the impact of such schools or the evidence of which configurations and elements of such schools are important. The study's focus is on the extent to which inclusive STEM high schools contribute to improved academic outcomes, interests in STEM careers, and expectations for post secondary study. The research study engages in implementation research to examine the elements of the STEM schools' design and implementation and other contextual factors, including state policies, which are associated with superior outcomes.
This is a quasi-experimental study of the effects of attending an inclusive STEM high school comparing outcomes for students in these schools with those of their counterparts attending other types of schools in the same states. The study includes all students in the 9th or 12th grade in the inclusive STEM high schools and students in samples of same-state comparison schools identified through propensity score matching. Data are collected longitudinally using student records and surveying students at regular intervals. The study follows the 12th grade students after graduation into postsecondary study and the workforce. The states identified in this study have the requisite administrative data systems to support the proposed study. By using a combination of data available in state-level data bases and new information obtained through project surveys, the researchers are identifying students who are matched not only on demographic variables and academic achievement before high school entry, but also on indicators of pre-existing interest and expectation such as self-efficacy and prior participation in informal STEM-related activities. Impacts on student achievement are analyzed separately for each state. Data on the elements of STEM schools are collected through teacher and administrator surveys and interviews. State STEM school history and policy data are collected through document analysis and interviews. The study utilizes hierarchical regression models, with separate models of each outcome measure and adjustments for tests of multiple comparisons. Student attrition is monitored and findings are examined to determine influence of attrition.
This project focuses on inclusive rather than selective STEM schools so that the population of students more typically represents the population of the students locally. The study provides a source of evidence about not only the effectiveness of STEM schools, but also contextual evidence of what works and for whom and under what conditions.
School Organization and Science Achievement: Organization and Leadership Influences on Equitable Student Performance
This project will document factors explaining variations in science achievement across schools enrolling ethnically and linguistically diverse students. The research question is: what leadership and organizational features at the school level are associated with mitigating science achievement gaps? At the conclusion of the five-year project, the findings will take the form of recommendations about leadership practices and school organization that can be implemented in other school settings.
The School Organization and Science Achievement (SOSA) Project will document factors explaining variations in science achievement across schools enrolling ethnically and linguistically diverse students. The research question is: what leadership and organizational features at the school level are associated with mitigating science achievement gaps? Previous school effectiveness studies demonstrate school leadership and social capital influencing student achievement; the SOSA project is unique with its focus on science achievement. Researchers at the University of Connecticut and the University of South Florida St. Petersburg, in collaboration with school districts in their respective states, will identify school leadership practices that can be connected with reductions in achievement gaps related to student ethnicity, English fluency, and social status. At the conclusion of the five-year project, the findings will take the form of recommendations about leadership practices and school organization that can be implemented in other school settings.
The project uses a mixed methods design by combining statistical modeling and qualitative data. Multiple regression analyses highlight those schools populated by fifth graders that have greater or lesser achievement gaps in science. Using social capital theory (i.e., school norms, communication channels, and trustworthiness) comparisons of positive and negative outlier schools will be made via interviews of building principals, classroom teachers and community representatives. The expectation is that schools providing more equitable science experiences to all students will exhibit stronger social capital compared to buildings with disparities in science test scores across demographic categories. These insights will be supplemented by multilevel structural equation modeling to determine the strength of association between various school climate measures (e.g., teacher-to-principal trust, correspondence between teacher and principal perceptions of leadership, and school/community ties) and science achievement as measured by statewide fifth grade science tests. In addition, growth analyses will be used to detect shifts over time and provide insights about the links between policy changes or leadership adjustments, inasmuch as science achievement gaps are affected.
By working with 150 schools in two states, this collaborative research project is designed to generate findings applicable in other school systems. Particularly in settings where science achievement gaps are large, and especially when such gaps vary between schools even when the student populations are similar, the findings from this study will have practical leadership implications. Expertise in this project includes science education, educational leadership, and statistical modeling. This complementary combination increases the depth of the project's efforts along with expanding its potential impacts. Key questions addressed by this project include: to what extent is leadership in science similar to or different from leadership in other subject areas? how do variations in leadership design (e.g., top-down versus distributed leadership) contribute to reductions in science achievement gaps? to what degree can effective leadership mitigate other factors that exacerbate the challenges of providing high quality science learning experiences for every child? Findings will be disseminated via the SOSA Project website, along with leadership development strategies. Deliverables include templates to replicate the study, case studies for professional development, and strategies for supporting the development of science teacher-leaders.
Investigating and Supporting the Development of Ambitious and Equitable Mathematics Instruction at Scale
This project is supporting and investigating the implementation of reformed mathematics instruction at the middle school level in two large school districts. The primary goal of the project is to develop an empirically grounded theory of action for implementing reform at school and district levels. The researchers are investigating reform within a coherent system that focuses on leadership and school-based professional development.
The Development of Ambitious and Equitable Mathematics Instruction project is supporting and investigating the implementation of reformed mathematics instruction at the middle school level in two large school districts. Project researchers are asking: What does it take to support mathematics teachers' development of ambitious and equitable instructional practices on a large scale? The project has built on what was learned in a previous, successful project studying the implementation of a middle school mathematics curriculum. The primary goal of the new project is to develop an empirically grounded theory of action for implementing reform at school and district levels. The researchers are investigating reform within a coherent system that focuses on leadership and school-based professional development. In addition, they are facilitating a longitudinal study of the curriculum implementation by continuing the data collection from the original study.
In order to build a theory of action, the project team is synthesizing data from a variety of domains including instructional systems (e.g., curriculum, materials, professional development, support for struggling students, and learning communities), mathematics coaching, networks of teachers, school leadership, and district leadership. Investigators are using a variety of analytic techniques to successfully integrate both quantitative and qualitative data as they seek to understand how school district strategies are playing out in schools and classrooms and how those strategies can be revised in order to improve student learning of mathematics.
An empirically grounded theory of action for implementing reform will help the mathematics education community to implement and to understand the process of reforming mathematics instruction at the middle school level. Many advances in mathematics instruction have been documented within a limited context, but researchers and practitioners need to understand the full range of action necessary to achieve similar successes at a district-wide level. The model developed from this project, in conjunction with longitudinal data, has the potential to guide future reform efforts that seek to provide ambitious and equitable mathematics instruction.
Beyond Bridging: Co-education of Pre-service and In-service Elementary Teachers in Science and Mathematics
This project will implement and study a professional community designed to alleviate the mismatch between the expectations of student teachers in mathematics and science and their mentor in-service teachers. The project is creating a neutral forum for the exchange of perspectives on issues of pedagogy with the expectation that student teachers would implement inquiry-based science and problem-solving mathematics pedagogies with the knowledgeable support of their mentor teachers.
The University of Arizona is partnering with the Tucson Unified School District to implement and study a professional community designed to alleviate the mismatch between the expectations of student teachers in mathematics and science and their mentor in-service teachers. This vexing problem often arises when student teachers expect to implement reform-based pedagogies while their mentor teachers insist on traditional approaches. The project is creating a "third space," a professional community that includes 40 pre-service and 50 in-service teachers, university scientists and mathematicians, science and mathematics education faculty, and school district administrators. The third space is providing a neutral forum for the exchange of perspectives on issues of pedagogy with the expectation that student teachers would implement inquiry-based science and problem-solving mathematics pedagogies with the knowledgeable support of their mentor teachers. The project is being implemented in two low-income, culturally and linguistically diverse elementary schools with a comparison school used as a control.
The evaluation/research component is a qualitative study led by Horizon Research, Inc. The fundamental research question is whether the third space model establishes interpretive systems that foster enactment of inquiry-based and problem-solving teaching practices. Data collection will include all participants in the third space forum, but focuses on the pre-service and in-service teachers through written products and discussions of lesson design activities, videotapes of teaching by pre-service and in-service teachers, and analysis of comments made in a web-based forum. Instruments to be used are the Reform Teaching Observation Protocol (RTOP), the Experiences Patterns Explanations (EPE) framework, and the Inquiry-Application Instructional Model (I-AIM).
The main product of this project is the third space model and the research that supports its success. The model will be disseminated broadly and if replicated widely, it would represent a major improvement in the professional development of teachers in the areas of inquiry-based science and problem-solving mathematics.