# Pedagogical Content Knowledge

## Proportions Playground: A Dynamic World to Support Teachers' Proportional Reasoning

This project focuses on the creation of the initial functionality for a dynamic microworld, Proportions Playground, designed to support teachers in developing a coherent understanding of proportional reasoning. The Proportions Playground project seeks to both develop a unique pilot software application for the iPad and explore how it supports teachers in developing a coherent, robust definition of proportions.

Award Number:
1621290
Funding Period:
Thu, 09/01/2016 to Thu, 02/28/2019
Full Description:

Proportions are a critical topic in mathematics that is simultaneously complicated and over-simplified in typical instruction. Current research undertaken by the research team suggests that the over-simplification is related to limitations in teachers' understandings of proportional relationships. Presenting proportions in a dynamic environment offers teachers the opportunity to create key developmental understandings related to this area of mathematics. This project focuses on the creation of the initial functionality for a dynamic microworld, Proportions Playground, designed to support teachers in developing a coherent understanding of proportional reasoning. Proportions Playground is conceptualized as a tool for supporting the development of coherent understandings by allowing teachers to interact in concrete ways with otherwise abstract ideas and by allowing teachers easy access to dynamic objects and other representations. It is meant to address the significant limitations for reasoning about the relationships between measurable aspects of two objects as well as in manipulating those relationships. Building from work currently underway, Proportions Playground will explore key areas in which there are opportunities for engaging teachers in the development of a coherent and robust understanding of proportional reasoning that extends beyond the typical "3 given, 1 unknown" proportion problem. This approach attempts to engage teachers in an array of dynamic, visually-rich sets of tasks designed to challenge teachers' preconceptions of proportions and to strengthen their connections between proportions and related areas of mathematics. This project is funded by the Discovery Research PreK-12 (DRK-12) and EHR Core Research (ECR) Programs. the DRK-12 program supports research and development on STEM education innovations and approaches to teaching, learning, and assessment. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field.

The Proportions Playground project seeks to both develop a unique pilot software application for the iPad and explore how it supports teachers in developing a coherent, robust definition of proportions. The software will be designed to support either numeric manipulation (e.g., graphing software) or geometric constructions (e.g., dynamic geometry software). Specifically, for this project the mathematics of interest will include the relationships between similarity and proportion and the nature of covariation. The research will focus on how teachers are developing a robust and coherent understanding of proportions and how the dynamic environment promotes such understandings. Working with six teacher advisors, the project will develop three task sets. Using teaching experiments and individual interviews, results will be used to refine the task sets. The revised task sets will be piloted with 40 teachers. Data will be collected on participants' thinking and any changes seen in the knowledge resources they are using. The researchers will be looking for factors that seem to impact teachers' thinking as well as evidence to support or deny the assertion that the Proportions Playground activities engage teachers in (a) different ways of reasoning about proportions and (b) support them in drawing from a wide array of resources so that coherence may be developed were the teachers to have a prolonged engagement with the tools. The project will rely on Epistemic Network Analysis to identify the connections between knowledge resources.

## Building a Next Generation Diagnostic Assessment and Reporting System within a Learning Trajectory-Based Mathematics Learning Map for Grades 6-8

This project will build on prior funding to design a next generation diagnostic assessment using learning progressions and other learning sciences research to support middle grades mathematics teaching and learning. The project will contribute to the nationally supported move to create, use, and apply research based open educational resources at scale.

Award Number:
1621254
Funding Period:
Thu, 09/15/2016 to Sat, 08/31/2019
Full Description:

This project seeks to design a next generation diagnostic assessment using learning progressions and other research (in the learning sciences) to support middle grades mathematics teaching and learning. It will focus on nine large content ideas, and associated Common Core State Standards for Mathematics. The PIs will track students over time, and work within school districts to ensure feasibility and use of the assessment system.

The research will build on prior funding by multiple funding agencies and address four major goals. The partnership seeks to address these goals: 1) revising and strengthening the diagnostic assessments in mathematics by adding new item types and dynamic tools for data gathering 2) studying alternative ways to use measurement models to assess student mathematical progress over time using the concept of learning trajectories, 3) investigating how to assist students and teachers to effectively interpret reports on math progress, both at the individual and the class level, and 4) engineering and studying instructional strategies based on student results and interpretations, as they are implemented within competency-based and personalized learning classrooms. The learning map, assessment system, and analytics are open source and can be used by other research and implementation teams. The project will exhibit broad impact due to the number of states, school districts and varied kinds of schools seeking this kind of resource as a means to improve instruction. Finally, the research project contributes to the nationally supported move to create, use, and apply research based open educational resources at scale.

## Supporting Instructional Growth in Mathematics: Enhancing Urban Secondary Teachers' Professional Learning through Formative Feedback

This project will explore the potential of video-based formative feedback to enhance professional development around ambitious instruction for secondary teachers in urban schools.

Award Number:
1620920
Funding Period:
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description:

Research continues to show the benefits of ambitious instruction for student learning of mathematics, yet ambitious instruction continues to be rare in U.S. schools, particularly in schools that serve historically marginalized students. Secondary teachers' learning and enactment of ambitious instruction in mathematics requires conceptual change, and their development could benefit from adequate and timely feedback close to classroom instruction. For this reason, the project will explore the potential of video-based formative feedback to enhance professional development. The focus of the partnership between university researchers and a well-regarded professional development organization, Math for America Los Angeles (MfA LA) will be on career-long learning of secondary mathematics teachers in urban schools. Results from this project will provide a theory of mathematics teachers' learning that can inform other instructional improvement efforts, with ecological validity in the critical site of urban schools. The framework and theory will be detailed at the level of specific tools and concrete practices that are learnable by teachers, school leaders, or instructional coaches. This project is funded by the Discovery Research Pre-K-12 Program, which funds research and development of STEM innovations and approaches in assessment, teaching and learning.

The question the project will address is: How can the project use formative feedback to enhance mathematics teachers' professional learning environments that support their development of ambitious instruction in urban schools? Formative feedback refers to tools and processes that ascertain teachers' current understandings and responsively adapt learning activities to better guide them toward their learning goals. Professional learning environments refer to the multiple sites of teachers' learning, from formal professional development activities to their school workplace. Ambitious instruction is defined as teaching approaches that aim to provide all students with ample opportunities to develop conceptual understanding of key mathematical ideas, participate in mathematical argumentation, connect multiple mathematical representations, as well as become fluent with mathematical procedures and processes. The persistence of typical mathematics instruction is framed as, in large part, an issue of teacher learning. Using design-based implementation research and interpretive methods, the project team will co-develop video-based formative assessment processes to guide teachers' evolving classroom practice.

## Improving the Implementation of Rigorous Instructional Materials in Middle Grades Mathematics: Developing a System of Practical Measures and Routines (Collaborative Research: Ahn)

The goal of this project is to improve the implementation of rigorous instructional materials in middle-grades mathematics at scale through a system of practical measures and routines for collecting and using data that both assesses and supports implementation.

Award Number:
1911492
Funding Period:
Sat, 10/01/2016 to Thu, 09/30/2021
Full Description:

The goal of this 5-year research project is to improve the implementation of rigorous instructional materials in middle-grades mathematics at scale. Many projects seek to improve mathematics instruction, but are not able to easily track their efforts at improvement. The primary product of this project will be a system of practical measures and routines for collecting and using data that both assesses and supports the implementation of rigorous instructional materials in middle-grades mathematics. In contrast to research and accountability measures, practical measures are assessments that require little time to administer and can thus be used frequently. The data can be analyzed rapidly so that teachers can receive prompt feedback on their progress, and instructional leaders can use the data to decide where to target resources to support improvement in the quality of instruction and student learning. The system of practical measures and routines will include 1) measures of high-leverage aspects of teachers' instructional practices that have been linked to student learning (e.g., rigor of tasks, quality of students' discourse) and attend to equitable student participation; and 2) measures of high-leverage aspects of key supports for improving the quality of teachers' practice (e.g., quality of professional development; coaching); and 3) a set of routines regarding how to use the resulting data to engage in rapid, improvement efforts. A key principle of the proposed project is that the system of measures and routines can be adapted to a wide range of school and district contexts. This project is supported by the Discovery Research preK-12 (DRK-12) program. The DRK-12 program supports research and development of STEM education innovations and approaches in assessment, learning, and teaching.

The project will establish three research-practice partnerships with five districts, in three different states, that are currently implementing rigorous instructional materials in middle-grades mathematics. Year 1 will focus on the development of a set of practical measures of classroom instruction. Year 2 will focus on testing the use of the classroom measures in the context of supports for teachers' learning, and the development of practical measures of key supports for teachers' learning. Years 3-4 will focus on how the project can "learn our way to scale" (Bryk et al., 2015), which requires strategically implementing measures and routines in increasingly diverse conditions. The project will engage in rapid improvement cycles in which researchers will work alongside district leaders and professional development (PD) facilitators to analyze the data from the measures of both classroom instruction and the quality of support for teacher learning to test the effectiveness of improvements in intended supports for teacher learning and to adjust the design of the support based on data. Across Years 1-4, the project will use recent developments in technology and information visualization to test and improve 1) the collection of practical measures in situ and 2) the design of data representations (or visualizations) that support teachers and leaders to make instructional improvement decisions. In Year 5, the project will conduct formal analyses of the relations between supports for teachers' learning; teachers' knowledge and classroom practices; and student learning.

This project was previously funded under award #1620900 and 1719744.

## Developing Formative Assessment Tools and Routines for Additive Reasoning

This design and development project is an expansion of the Ongoing Assessment Project (OGAP), an established model for research-based formative assessment in grades 3-8, to the early elementary grades. The project will translate findings from research on student learning of early number, addition, and subtraction into tools and routines that teachers can use to formatively assess their students' understanding on a regular basis and develop targeted instructional responses.

Award Number:
1620888
Funding Period:
Thu, 09/01/2016 to Thu, 02/28/2019
Full Description:

This design and development project is an expansion of the Ongoing Assessment Project (OGAP), an established model for research-based formative assessment in grades 3-8, to the early elementary grades. OGAP brings together two powerful ideas in mathematics education - formative assessment and research based learning trajectories - to enhance teacher knowledge, instructional practices, and student learning. Building on a proven track record of success with this model, the current project will translate findings from research on student learning of early number, addition, and subtraction into tools and routines that teachers can use to formatively assess their students' understanding on a regular basis and develop targeted instructional responses. The project involves a development component focused on producing and field testing new resources (including frameworks, item banks, pre-assessments and professional development materials) and a research component designed to improve the implementation of these resources in school settings. The materials that are developed from this project will help teachers be able to more precisely assess student understanding in the major mathematical work of grades K-2 in order to better meet the needs of diverse learners. With the addition of these new early elementary materials, OGAP formative assessment resources will be available for use from kindergarten through grade 8.

Although much attention has been paid to the improvement of early literacy, building strong mathematical foundations and early computational fluency is equally critical for later success in school and preparation for STEM careers. This project will develop and field test tools, resources, and routines that teachers can employ to help young students develop deeper conceptual understandings and more powerful and efficient strategies in the early grades. The project emerged from the needs of school-based practitioners looking for instructional support in the primary grades and uses design-based research methodology. The new materials will be developed, tested, and revised through multiple iterations of implementation in schools. Research-based learning trajectories will be consolidated into simplified frameworks that illustrate the overall progression of major levels of student thinking in the domains of counting, addition, and subtraction. A bank of formative assessment items will be developed, field tested, and refined through a three-phase validation process. Professional development modules will be designed and field tested to support teacher knowledge and effective use of the formative assessment tools and routines. Data collected on key activities in the formative assessment process (including teacher selection of items, analysis of student work, instructional implications, and enacted instructional response) will be used to continually inform development as well as illuminate the conditions under which formative assessment leads to productive changes in instruction and student learning in the classroom. The project will yield a set of field tested tools and resources ready for both broader dissemination and further research on the promise of the intervention, as well as an understanding of how to support effective implementation.

## Developing Preservice Elementary Teachers' Ability to Facilitate Goal-Oriented Discussions in Science and Mathematics via the Use of Simulated Classroom Interactions

The project will develop, pilot, and validate eight discussion-oriented performance tasks that will be embedded in an online simulated classroom environment. The resulting research and development products could be used nationwide in teacher preparation and professional development settings to assess and develop teachers' ability to support classroom discussion in science and mathematics.

Award Number:
1621344
Funding Period:
Mon, 08/01/2016 to Fri, 07/31/2020
Full Description:

There is widespread recognition in educational literatures that academic discourse is important for supporting students' developing understanding in the disciplines of science and mathematics. College and career-ready standards also call for attention to supporting students' learning of how to think and communicate like disciplinary experts. The teaching practice of orchestrating classroom discussion is intended to support students in obtaining higher levels of academic achievement but also to support students' participation in a democratic society. However, research has found that teachers--particularly novice teachers--struggle to orchestrate discussion effectively for science and mathematics. The investigators of this project hypothesize that opportunities to 1) practice orchestrating discussions in simulated classroom environments; 2) receive constructive feedback on their practice; and 3) reflect on that feedback and their experiences with peers and teacher educators, develops preservice teachers' abilities to lead productive classroom discussion. This may allow them to be more effective at orchestrating discussion when they begin teaching real students in science and mathematics classrooms. The project team, which includes investigators from Educational Testing Service (ETS) and software engineers at Mursion, will develop, pilot, and validate eight discussion-oriented performance tasks that will be embedded in an online simulated classroom environment. The resulting research and development products could be used nationwide in teacher preparation and professional development settings to assess and develop teachers' ability to support classroom discussion in science and mathematics.

The Discovery Research K-12 (DRK-12) program seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models, and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This Early Stage Design and Development project will 1) iteratively develop, pilot, and refine eight science and mathematics discussion-oriented performance tasks (six formative, two summative), scoring rubrics, and rater training materials; 2) deploy the intervention in four university sites, collecting data from 240 prospective teachers in both treatment and business-as-usual courses; and 3) use data analyses and expert review to build a five-part argument for the validity of the assessment and scoring rubrics. Data sources include prospective teachers' background and demographic information, cognitive interviews, surveys, scores on content knowledge for teaching (CKT) instruments, performance and scores on the developed performance tasks, discussion scores on Danielson's Framework for Teaching observation protocol, and case study interviews with prospective teachers. The project team will also conduct interviews with teacher educators and observe classroom debrief sessions with prospective teachers and their teacher educators. The research will examine each teacher's scores on two summative performance tasks administered pre- and post-intervention and will look for evidence of growth across three formative tasks. Linear regression models will be used to understand relationships among teachers' CKT scores, pre-intervention performance task scores, group assignment, and post-intervention performance task scores. A grounded theory approach to coding qualitative data of 24 case study teachers, observations of debrief sessions, and interviews with teacher educators will generate descriptive use cases, illustrating how the tools can support prospective teachers in learning how to facilitate discussions focused on science and mathematics argumentation. Mursion will develop a webpage on its website dedicated to this project that will allow the team to post the new performance-based tasks, scoring rubrics, and examples of performance in the simulated environment for teacher educators, educational researchers, and policy makers and collect feedback from them that can be used as another information source for refining tools and their use. Research findings will also be disseminated by more traditional means, such as papers in peer-reviewed research and practitioner journals and conference presentations.

## Project Videos

2020 STEM for All Video Showcase

 Title: Tales from the Sandbox: Learning through Simulated Teaching Presenter(s): Heather Howell, Jamie Mikeska, & Carrie Straub

2019 STEM for All Video Showcase

 Title: Simulated Classrooms as Practice-Based Learning Spaces Presenter(s): Jamie Mikeska, Heather Howell, & Carrie Straub

2018 STEM for All Video Showcase

 Title: Leading Science/Math Discussions in a Simulated Classroom Presenter(s): Heather Howell, Jamie Mikeska, & Carrie Straub
2017 STEM for All Video Showcase
 Title: Simulated Classroom Environments for Discussions Presenter(s): Jamie Mikeska, Heather Howell, & Carrie Straub

## CAREER: Multilevel Mediation Models to Study the Impact of Teacher Development on Student Achievement in Mathematics

This project will develop a comprehensive framework to inform and guide the analytic design of teacher professional development studies in mathematics. An essential goal of the research is to advance a science of teaching and learning in ways that traverse both research and education.

Award Number:
1552535
Funding Period:
Thu, 09/01/2016 to Tue, 08/31/2021
Full Description:

This is a Faculty Early Career Development Program (CAREER) project. The CAREER program is a National Science Foundation-wide activity that offers the most prestigious awards in support of junior faculty who exemplify the role of teacher-scholars through outstanding research, excellent education and the integration of education and research. The intellectual merit and broader impacts of this study lie in two complementary contributions of the project. First, the development of the statistical framework for the design of multilevel mediation studies has significant potential for broad impact because it develops a core platform that is transferable to other STEM (science, technology, engineering, and mathematics) education areas and STEM disciplines. Second, the development of software and curricular materials to implement this framework further capitalize on the promise of this work because it distributes the results in an accessible manner to diverse sets of research and practitioner groups across STEM education areas and STEM disciplines. Together, the components of this project will substantially expand the scope and quality of evidence generated through mathematics professional development and, more generally, multilevel mediation studies throughout STEM areas by increasing researchers' capacity to design valid and comprehensive studies of the theories of action and change that underlie research programs.

This project will develop a comprehensive framework to inform and guide the analytic design of teacher professional development studies in mathematics. The proposed framework incorporates four integrated research and education components: (1) develop statistical formulas and tools to guide the optimal design of experimental and non-experimental multilevel mediation studies in the presence of measurement error, (2) develop empirical estimates of the parameters needed to implement these formulas to design teacher development studies in mathematics, (3) develop free and accessible software to execute this framework, and (4) develop training materials and conduct workshops on the framework to improve the capacity of the field to design effective and efficient studies of teacher development. An essential goal of the research is to advance a science of teaching and learning in ways that traverse both research and education.

## Science, Technology, Engineering and Mathematics Scholars Teacher Academy Resident System

This project will investigate the effectiveness of a teacher academy resident model to recruit, license, induct, employ, and retain middle school and secondary teachers for high-need schools in the South. It will prepare new, highly-qualified science and mathematics teachers from historically Black universities in high-needs urban and rural schools with the goal of increasing teacher retention and diversity rates.

Partner Organization(s):
Award Number:
1621325
Funding Period:
Fri, 07/15/2016 to Wed, 06/30/2021
Full Description:

This project at Jackson State University will investigate the effectiveness of a teacher academy resident model to recruit, license, induct, employ, and retain middle school and secondary science and mathematics teachers for high-need schools in the South. It will prepare new, highly-qualified science and mathematics teachers from historically Black universities in high-needs urban and rural schools. The project involves a partnership among three historically Black universities (Jackson, State University, Xavier University of Louisiana, and the University of Arkansas at Pine Bluff), and diverse urban and rural school districts in Jackson, Mississippi; New Orleans, Louisiana; and Pine Bluff Arkansas region that serve more than 175,000 students.

Participants will include 150 middle and secondary school teacher residents who will gain clinical mentored experience and develop familiarity with local schools. The 150 teacher residents supported by the program to National Board certification will obtain: state licensure/certification in science teaching, a master's degree, and initiation. The goal is to increase teacher retention and diversity rates. The research question guiding this focus is: Will training STEM graduates have a significant effect on the quality of K-12 instruction, teacher efficacy and satisfaction, STEM teacher retention, and students? Science and mathematics achievement? A quasi-experimental design will be used to evaluate project's effectiveness.

## Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum (Collaborative Research: Greenwald)

The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Partner Organization(s):
Award Number:
1621441
Funding Period:
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description:

This is an early-stage design and development collaborative study submitted to the assessment strand of the Discovery Research PreK-12 (DRK-12) program, in response to Program Solicitation NSF 15-592. The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. To achieve this purpose, the project will examine the validity of a new technology-based formative assessment tool for classroom argumentation--"Diagnosing the Argumentation Levels of Groups" (DiALoG)--for which psychometric validation work has been conducted in a laboratory setting. The DiALoG assessment tool allows teachers to document classroom talk and display scores across multiple dimensions--both intrapersonal and interpersonal--for formative assessment purposes. The project will work with 6th-8th grade science teachers to monitor and support argumentation through real-time formative assessment data generated by the DiALoG instrument. DiALoG will be used in conjunction with "Amplify Science", a Lawrence Hall of Science-developed curriculum that incorporates the science practice of engaging in argument from evidence, and a suite of newly developed Responsive Mini-Lessons (RMLs), which consist of 20-30 minute instructional strategies designed to assist teachers to provide feedback to students' thinking and follow-up to argumentation episodes that the DiALoG tool identifies in need of further support. The study will allow the refinement and expansion of DiALoG and evaluation of its impact on teacher pedagogical content knowledge and formative assessment practices in widespread classroom use.

The project will address two specific research questions: (1) How can DiALoG be refined to provide a formative assessment tool for oral argumentation that is reliable, practical, and useful in middle school classrooms?; and (2) How does the use of DiALoG affect teacher formative assessment practices around evidence-based argumentation, when implementing science units designed to support oral argumentation? In order to answer these questions, the project will conduct a randomized control trial with 100 teachers: 50 will teach argumentation-focused curriculum with DiALoG, 50 will teach the same curriculum without DiALoG. Both control and treatment teachers will receive all digital and physical materials needed to teach three Amplify Science curriculum units. Treatment teachers will be provided also with the most recent version of DiALoG, including the linked RMLs, as well as support materials for using DiALoG with the Amplify curriculum. A subgroup of focus teachers (5 from the treatment group, and 5 from the control group) will be the subject of additional data collection and analysis. Three focus lessons, in which students are engaging in small-group or whole-class oral argumentation, will be selected from each of the three Amplify Science curricular units. Teacher measures for the randomized control trial will include validated instruments, such as (a) a pre- and post-assessment of teacher pedagogical content knowledge; (b) post-lesson and post-unit surveys in which teachers will self-report on their formative assessment practices; and (c) video recordings of selected lessons in the focus classrooms. In order to observe potential differences in formative assessment practices between treatment and control, protocols will be used to analyze the video recordings of focus classrooms, including (a) Reformed Teaching Observation Protocol; (b) Assessment of Scientific Argumentation inside the Classroom; and (c) Formative Assessment for Teachers and Students. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

## Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum (Collaborative Research: Henderson)

The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.