Mixed Methods

Development and Empirical Recovery for a Learning Progression-Based Assessment of the Function Concept

The project will design an assessment based on learning progressions for the concept of function - a critical concept for algebra learning and understanding. The goal of the assessment and learning progression design is to specifically incorporate findings about the learning of students traditionally under-served and under-performing in algebra courses.

Lead Organization(s): 
Award Number: 
1621117
Funding Period: 
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description: 

The project will design an assessment based on learning progressions for the concept of function. A learning progression describes how students develop understanding of a topic over time. Function is a critical concept for algebra learning and understanding. The goal of the assessment and learning progression design in this project is to specifically incorporate findings about the learning of students traditionally under-served and under-performing in algebra courses. The project will include accounting for the social and cultural experiences of the middle and high school students when creating assessment tasks. The resources developed should impact mathematics instruction (especially for algebra courses) by creating a learning progression which captures the range of student performance and appropriately places them at distinct levels of performance. The important contribution of the work is the development of a learning progression and related assessment tasks that account for the experiences of students often under-served in mathematics. The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

The learning progression development will begin by comparing and integrating existing learning progressions and current research on function learning. This project will develop an assessment of student knowledge of function based on learning progressions via empirical recovery (looking for the reconstruction of theoretical levels of the learning theory). Empirical recovery is the process through which data will be collected that reconstruct the various levels, stages, or sequences of said learning progression. The development of tasks and task models will include testing computer-delivered, interactive tasks and rubrics that can be used for human and automated scoring (depending on the task). Item response theory methods will be used to evaluate the assessment tasks' incorporation of the learning progression.


Project Videos

2019 STEM for All Video Showcase

Title: Concept of Function Learning Progression

Presenter(s): Edith Graf, Frank Davis, Chad Milner, Maisha Moses, & Sarah Ohls


Organizing to Learn Practice: Teacher Learning in Classroom-Focused Professional Development

This project addresses the fundamental challenge of how to support teachers to improve their practice. The approach uses a "live mathematics classroom" as a common text for working on practice, where participants are not only watching and discussing but are engaged in developing and learning practice. The project will generate new knowledge regarding ways in which elementary teachers of mathematics can be supported to learn effective teaching practice.

Lead Organization(s): 
Award Number: 
1621104
Funding Period: 
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description: 

Growing evidence about the powerful effects of skillful teaching on students' learning creates a need to for professional development that impacts teachers' actual practice. Just as other professions (e.g., nursing, social work, law) have centered practitioners' learning in "live" practice with structures that support learning in context, the project will investigate whether and how this can be accomplished in teaching. The approach uses a "live mathematics classroom" as a common text for working on practice, where participants are not only watching and discussing but are engaged in developing and learning practice. The project also explores the following variations in practice-based professional development: (1) on-site and remote participation of teachers; and (2) the addition of supplementary practice-focused professional development. The project will generate new knowledge regarding ways in which elementary teachers of mathematics can be supported to learn effective teaching practice.

This project addresses a fundamental challenge for professional development, that is, how to support teachers to improve their practice. Teachers profit from well-designed opportunities to develop new visions for practice, learn more about students' thinking, or work on specific mathematical topics or tasks. Still, such opportunities are often insufficient to support teachers with the complexity of classroom teaching. These kinds of professional opportunities focus on critical resources for instruction but not on the details of teaching practice itself. This practice-centered professional development is situated within a summer mathematics program for fifth graders. The proposed research will explore the impact on teachers' practice, as well as on their knowledge and dispositions, from participating in these structured ways. Three studies will resolve the following three sets of questions: (1) What do teachers learn from structured participation in the class? Does their participation impact their own teaching practice, and if so, in what ways? (2) Does the setting of the peripheral participation matter? Does this form of participation impact their own teaching practice, and if so, in what ways? (3) Does the addition of professional development focused on a particular teaching practice impact teachers' own practice, and if so, in what ways? How does the addition of professional development focused on a specific instructional practice compare across the in-person and online forms of participation in terms of impact on teachers' own practice? The project will collect and analyze several types of data pre- and post-intervention, including measures of mathematical knowledge for teaching, measures of language for talking about the work of teaching and students, and skill with leading a mathematics discussion, and the mathematical quality of instruction. The project will generate new knowledge related to to organizing professional learning around supports that teachers need to learn practice as well as ways to study their learning of teaching practice.

Modest Supports for Sustaining Professional Development Outcomes over the Long-Term

This study will investigate factors influencing the persistence of teacher change after professional development (PD) experiences, and will examine the extent to which modest supports for science teaching in grades K-5 sustain PD outcomes over the long term.

Lead Organization(s): 
Award Number: 
1620979
Funding Period: 
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description: 

This study will investigate factors influencing the persistence of teacher change after professional development (PD) experiences, and will examine the extent to which modest supports for science teaching in grades K-5 sustain PD outcomes over the long term. Fifty K-12 teachers who completed one of four PD programs situated in small, rural school districts will be recruited for the study, and they will participate in summer refresher sessions for two days, cluster meetings at local schools twice during the academic year, and optional Webinar sessions two times per year. Electronic supports for participants will include a dedicated email address, a project Facebook page, a biweekly newsletter, and access to archived Webinars on a range of topics related to teaching elementary school science. Modest support for replacement of consumable supplies needed for hands-on classroom activities will also be provided. The project will examine the extent to which these modest supports individually and collectively foster the sustainability of PD outcomes in terms of the instructional time devoted to science, teacher self-efficacy in science, and teacher use of inquiry-based instructional strategies. The effects of contextual factors on sustainability of PD outcomes will also be examined.

This longitudinal study will seek answers to three research questions: 1) To what extent do modest supports foster the sustainability of professional development outcomes in: a) instructional time in science; b) teachers' self-efficacy in science; and c) teachers' use of inquiry-based instructional strategies? 2) Which supports are: a) the most critical for sustainability of outcomes; and b) the most cost-effective; and 3) What contextual factors support or impede the sustainability of professional development outcomes? The project will employ a mixed-methods research design to examine the effects of PD in science among elementary schoolteachers over a 10 to 12 year period that includes a 3-year PD program, a 4-6 year span after the initial PD program, and a 3-year intervention of modest supports. Quantitative and qualitative data will be collected from multiple sources, including: a general survey of participating teachers regarding their beliefs about science, their instructional practices, and their instructional time in science; a teacher self-efficacy measure; intervention feedback surveys; electronic data sources associated with Webinars; teacher interviews; school administrator interviews; and receipts for purchases of classroom supplies. Quantitative data from the teacher survey and self-efficacy measure will be analyzed using hierarchical modeling to examine growth rates after the original PD and the change in growth after the provision of modest supports. Data gathered from other sources will be tracked, coded, and analyzed for each teacher, and linked to the survey and self-efficacy data for analysis by individual teacher, by grade level, by school, by district, and by original PD experience. Together, these data will enable the project team to address the project's research questions, with particular emphasis on determining the extent to which teachers make use of the various supports offered, and identifying the most cost-effective and critical supports.

An Online STEM Career Exploration and Readiness Environment for Opportunity Youth

This project aims to create a web-based STEM Career Exploration and Readiness Environment (CEE-STEM) that will support opportunities for youth ages 16-24 who are neither in school nor are working, in rebuilding engagement in STEM learning and developing STEM skills and capacities relevant to diverse postsecondary education/training and employment pathways.

Award Number: 
1620904
Funding Period: 
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description: 

CAST, the University of Massachusetts-Amherst, and YouthBuild USA aim to create a web-based STEM Career Exploration and Readiness Environment (CEE-STEM). This will support opportunities for youth ages 16-24 who are neither in school nor are working, in rebuilding engagement in STEM learning and developing STEM skills and capacities relevant to diverse postsecondary education/training and employment pathways. The program will provide opportunity youth with a personalized and portable tool to explore STEM careers, demonstrate their STEM learning, reflect on STEM career interests, and take actions to move ahead with STEM career pathways of interest.

The proposed program addresses two critical and interrelated aspects of STEM learning for opportunity youth: the development of STEM foundational knowledge; and STEM engagement, readiness and career pathways. These aspects of STEM learning are addressed through an integrated program model that includes classroom STEM instruction; hands-on job training in career pathways including green construction, health care, and technology.


Project Videos

2019 STEM for All Video Showcase

Title: Building a Diverse STEM Talent Pipeline: Finding What Works

Presenter(s): Tracey Hall

2018 STEM for All Video Showcase

Title: Bridging the Gap Between Ability and Opportunity in STEM

Presenter(s): Sam Johnston


Improving the Implementation of Rigorous Instructional Materials in Middle Grades Mathematics: Developing a System of Practical Measures and Routines (Collaborative Research: Ahn)

The goal of this project is to improve the implementation of rigorous instructional materials in middle-grades mathematics at scale through a system of practical measures and routines for collecting and using data that both assesses and supports implementation.

Award Number: 
1911492
Funding Period: 
Sat, 10/01/2016 to Thu, 09/30/2021
Full Description: 

The goal of this 5-year research project is to improve the implementation of rigorous instructional materials in middle-grades mathematics at scale. Many projects seek to improve mathematics instruction, but are not able to easily track their efforts at improvement. The primary product of this project will be a system of practical measures and routines for collecting and using data that both assesses and supports the implementation of rigorous instructional materials in middle-grades mathematics. In contrast to research and accountability measures, practical measures are assessments that require little time to administer and can thus be used frequently. The data can be analyzed rapidly so that teachers can receive prompt feedback on their progress, and instructional leaders can use the data to decide where to target resources to support improvement in the quality of instruction and student learning. The system of practical measures and routines will include 1) measures of high-leverage aspects of teachers' instructional practices that have been linked to student learning (e.g., rigor of tasks, quality of students' discourse) and attend to equitable student participation; and 2) measures of high-leverage aspects of key supports for improving the quality of teachers' practice (e.g., quality of professional development; coaching); and 3) a set of routines regarding how to use the resulting data to engage in rapid, improvement efforts. A key principle of the proposed project is that the system of measures and routines can be adapted to a wide range of school and district contexts. This project is supported by the Discovery Research preK-12 (DRK-12) program. The DRK-12 program supports research and development of STEM education innovations and approaches in assessment, learning, and teaching.

The project will establish three research-practice partnerships with five districts, in three different states, that are currently implementing rigorous instructional materials in middle-grades mathematics. Year 1 will focus on the development of a set of practical measures of classroom instruction. Year 2 will focus on testing the use of the classroom measures in the context of supports for teachers' learning, and the development of practical measures of key supports for teachers' learning. Years 3-4 will focus on how the project can "learn our way to scale" (Bryk et al., 2015), which requires strategically implementing measures and routines in increasingly diverse conditions. The project will engage in rapid improvement cycles in which researchers will work alongside district leaders and professional development (PD) facilitators to analyze the data from the measures of both classroom instruction and the quality of support for teacher learning to test the effectiveness of improvements in intended supports for teacher learning and to adjust the design of the support based on data. Across Years 1-4, the project will use recent developments in technology and information visualization to test and improve 1) the collection of practical measures in situ and 2) the design of data representations (or visualizations) that support teachers and leaders to make instructional improvement decisions. In Year 5, the project will conduct formal analyses of the relations between supports for teachers' learning; teachers' knowledge and classroom practices; and student learning.

This project was previously funded under award #1620900 and 1719744.

 

Improving the Implementation of Rigorous Instructional Materials in Middle-Grades Mathematics: Developing a System of Practical Measures and Routines (Collaborative Research: Jackson)

The goal of this project is to improve the implementation of rigorous instructional materials in middle-grades mathematics at scale through a system of practical measures and routines for collecting and using data that both assesses and supports implementation.

Lead Organization(s): 
Award Number: 
1620851
Funding Period: 
Sat, 10/01/2016 to Thu, 09/30/2021
Full Description: 

The goal of this 5-year research project is to improve the implementation of rigorous instructional materials in middle-grades mathematics at scale. Many projects seek to improve mathematics instruction, but are not able to easily track their efforts at improvement. The primary product of this project will be a system of practical measures and routines for collecting and using data that both assesses and supports the implementation of rigorous instructional materials in middle-grades mathematics. In contrast to research and accountability measures, practical measures are assessments that require little time to administer and can thus be used frequently. The data can be analyzed rapidly so that teachers can receive prompt feedback on their progress, and instructional leaders can use the data to decide where to target resources to support improvement in the quality of instruction and student learning. The system of practical measures and routines will include 1) measures of high-leverage aspects of teachers' instructional practices that have been linked to student learning (e.g., rigor of tasks, quality of students' discourse) and attend to equitable student participation; and 2) measures of high-leverage aspects of key supports for improving the quality of teachers' practice (e.g., quality of professional development; coaching); and 3) a set of routines regarding how to use the resulting data to engage in rapid, improvement efforts. A key principle of the proposed project is that the system of measures and routines can be adapted to a wide range of school and district contexts. This project is supported by the Discovery Research preK-12 (DRK-12) program. The DRK-12 program supports research and development of STEM education innovations and approaches in assessment, learning, and teaching.

The project will establish three research-practice partnerships with five districts, in three different states, that are currently implementing rigorous instructional materials in middle-grades mathematics. Year 1 will focus on the development of a set of practical measures of classroom instruction. Year 2 will focus on testing the use of the classroom measures in the context of supports for teachers' learning, and the development of practical measures of key supports for teachers' learning. Years 3-4 will focus on how the project can "learn our way to scale" (Bryk et al., 2015), which requires strategically implementing measures and routines in increasingly diverse conditions. The project will engage in rapid improvement cycles in which researchers will work alongside district leaders and professional development (PD) facilitators to analyze the data from the measures of both classroom instruction and the quality of support for teacher learning to test the effectiveness of improvements in intended supports for teacher learning and to adjust the design of the support based on data. Across Years 1-4, the project will use recent developments in technology and information visualization to test and improve 1) the collection of practical measures in situ and 2) the design of data representations (or visualizations) that support teachers and leaders to make instructional improvement decisions. In Year 5, the project will conduct formal analyses of the relations between supports for teachers' learning; teachers' knowledge and classroom practices; and student learning.

Developing A Discourse Observation Tool and Online Professional Development to Promote Science, Oral Language and Literacy Development from the Start of School

The goal of this project is to develop a classroom observation tool and an online professional development model to help early-elementary teachers improve science instruction among young learners by cultivating scientific discourse.

Lead Organization(s): 
Award Number: 
1620580
Funding Period: 
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description: 

The goal of this project is to develop resources and a professional development model to help early-elementary teachers improve science instruction among young learners by cultivating scientific discourse. A central component of the Next Generation Science Standards (NGSS) is engaging students in discourse with a focus on formulating and communicating scientific explanations. This project will develop a classroom observation tool that will help teachers examine changes in the quantity and quality of science discourse in K-2 classrooms over time. The project will also develop an online professional development (PD) model that uses the new observation tool to help teachers analyze their own classroom practices and the practice of others to improve classroom efforts to foster improved scientific discourse.

This early stage design and development study will employ a Design-Based Implementation Research (DBIR) approach to develop the new classroom observation tool and online professional development model, and then seek answers to the following research questions: 1) How can a classroom observation measure be developed to effectively capture the range in quality of science discourse in early elementary classrooms?; 2) How can an online PD model be developed based on the new observation tool?; 3) How do teachers' knowledge and instructional practice change over the course of participation in the yearlong PD?; and 4) How does the quantity and quality of science discourse change in K-2 classrooms over the course of teachers' participation in a yearlong online PD experience that is built around the new observation tool? The project will engage 36 teachers and their 36 different classrooms in Michigan and use multiple data sources to understand whether and how teacher knowledge and instructional practices change during participation in the new PD model. Multiple iterations of design, data collection, and refinement will be used to understand how, when, and why features of the PD and observation tool might combine to transform science discourse in early elementary classrooms. In years 3 and 4, the project team will conduct two year-long implementation trials with cohorts of 15 teachers and 5 instructional coaches (experienced science teachers) who will use the PD and tool in order study their implementation and make iterative improvements. The project will also gather data to understand changes in teacher knowledge and practice as well as video data to document changes in classroom discourse.

Developing Preservice Elementary Teachers' Ability to Facilitate Goal-Oriented Discussions in Science and Mathematics via the Use of Simulated Classroom Interactions

The project will develop, pilot, and validate eight discussion-oriented performance tasks that will be embedded in an online simulated classroom environment. The resulting research and development products could be used nationwide in teacher preparation and professional development settings to assess and develop teachers' ability to support classroom discussion in science and mathematics.

Lead Organization(s): 
Award Number: 
1621344
Funding Period: 
Mon, 08/01/2016 to Fri, 07/31/2020
Full Description: 

There is widespread recognition in educational literatures that academic discourse is important for supporting students' developing understanding in the disciplines of science and mathematics. College and career-ready standards also call for attention to supporting students' learning of how to think and communicate like disciplinary experts. The teaching practice of orchestrating classroom discussion is intended to support students in obtaining higher levels of academic achievement but also to support students' participation in a democratic society. However, research has found that teachers--particularly novice teachers--struggle to orchestrate discussion effectively for science and mathematics. The investigators of this project hypothesize that opportunities to 1) practice orchestrating discussions in simulated classroom environments; 2) receive constructive feedback on their practice; and 3) reflect on that feedback and their experiences with peers and teacher educators, develops preservice teachers' abilities to lead productive classroom discussion. This may allow them to be more effective at orchestrating discussion when they begin teaching real students in science and mathematics classrooms. The project team, which includes investigators from Educational Testing Service (ETS) and software engineers at Mursion, will develop, pilot, and validate eight discussion-oriented performance tasks that will be embedded in an online simulated classroom environment. The resulting research and development products could be used nationwide in teacher preparation and professional development settings to assess and develop teachers' ability to support classroom discussion in science and mathematics.

The Discovery Research K-12 (DRK-12) program seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models, and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This Early Stage Design and Development project will 1) iteratively develop, pilot, and refine eight science and mathematics discussion-oriented performance tasks (six formative, two summative), scoring rubrics, and rater training materials; 2) deploy the intervention in four university sites, collecting data from 240 prospective teachers in both treatment and business-as-usual courses; and 3) use data analyses and expert review to build a five-part argument for the validity of the assessment and scoring rubrics. Data sources include prospective teachers' background and demographic information, cognitive interviews, surveys, scores on content knowledge for teaching (CKT) instruments, performance and scores on the developed performance tasks, discussion scores on Danielson's Framework for Teaching observation protocol, and case study interviews with prospective teachers. The project team will also conduct interviews with teacher educators and observe classroom debrief sessions with prospective teachers and their teacher educators. The research will examine each teacher's scores on two summative performance tasks administered pre- and post-intervention and will look for evidence of growth across three formative tasks. Linear regression models will be used to understand relationships among teachers' CKT scores, pre-intervention performance task scores, group assignment, and post-intervention performance task scores. A grounded theory approach to coding qualitative data of 24 case study teachers, observations of debrief sessions, and interviews with teacher educators will generate descriptive use cases, illustrating how the tools can support prospective teachers in learning how to facilitate discussions focused on science and mathematics argumentation. Mursion will develop a webpage on its website dedicated to this project that will allow the team to post the new performance-based tasks, scoring rubrics, and examples of performance in the simulated environment for teacher educators, educational researchers, and policy makers and collect feedback from them that can be used as another information source for refining tools and their use. Research findings will also be disseminated by more traditional means, such as papers in peer-reviewed research and practitioner journals and conference presentations.


Project Videos

2019 STEM for All Video Showcase

Title: Simulated Classrooms as Practice-Based Learning Spaces

Presenter(s): Jamie Mikeska, Heather Howell, & Carrie Straub

2018 STEM for All Video Showcase

Title: Leading Science/Math Discussions in a Simulated Classroom

Presenter(s): Heather Howell, Jamie Mikeska, & Carrie Straub

 2017 STEM for All Video Showcase
Title: Simulated Classroom Environments for Discussions

Presenter(s): Jamie Mikeska, Heather Howell, & Carrie Straub


Developing Teachers as Computational Thinkers Through Supported Authentic Experiences in Computing Modeling and Simulation

This project addresses the need for a computationally-enabled STEM workforce by equipping teachers with the skills necessary to prepare students for future endeavors as computationally-enabled scientists and citizens, and by investigating the most effective ways to provide this instruction to teachers. The project also addresses the immediate challenge presented by NGSS to prepare middle school science teachers to implement rich computational thinking experiences within science classes.

Partner Organization(s): 
Award Number: 
1639069
Funding Period: 
Fri, 01/01/2016 to Sun, 06/30/2019
Full Description: 

The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

This project addresses the need for a computationally-enabled STEM workforce by equipping teachers with the skills necessary to prepare students for future endeavors as computationally-enabled scientists and citizens, and by investigating the most effective ways to provide this instruction to teachers. The project also addresses the immediate challenge presented by the Next Generation Science Standards to prepare middle school science teachers to implement rich computational thinking (CT) experiences, such as the use, creation and analysis of computer models and simulations, within science classes.

The project, a partnership between the Santa Fe Institute and the Santa Fe Public School District, directly addresses middle school teachers' understanding, practice, and teaching of modern scientific practice. Using the Project GUTS program and professional development model as a foundation, this project will design and develop a set of Resources, Models, and Tools (RMTs) that collectively form the basis for a comprehensive professional development (PD) program, then study teachers' experiences with the RMTs and assess how well the RMTs prepared teachers to implement the curriculum. The PD program includes: an online PD network; workshops; webinars and conferences; practicum and facilitator support; and curricular and program guides. The overall approach to the project is design based implementation research (DBIR). Methods used for the implementation research includes: unobtrusive measures such as self-assessment sliders and web analytics; the knowledge and skills survey (KS-CT); interviews (teachers and the facilitators); analysis of teacher modified and created models; and observations of practicum and classroom implementations. Data collection and analysis in the implementation research serve two purposes: a) design refinement and b) case study development. The implementation research employs a mixed-method, nonequivalent group design with embedded case studies.

Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum (Collaborative Research: Greenwald)

The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Partner Organization(s): 
Award Number: 
1621441
Funding Period: 
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description: 

This is an early-stage design and development collaborative study submitted to the assessment strand of the Discovery Research PreK-12 (DRK-12) program, in response to Program Solicitation NSF 15-592. The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. To achieve this purpose, the project will examine the validity of a new technology-based formative assessment tool for classroom argumentation--"Diagnosing the Argumentation Levels of Groups" (DiALoG)--for which psychometric validation work has been conducted in a laboratory setting. The DiALoG assessment tool allows teachers to document classroom talk and display scores across multiple dimensions--both intrapersonal and interpersonal--for formative assessment purposes. The project will work with 6th-8th grade science teachers to monitor and support argumentation through real-time formative assessment data generated by the DiALoG instrument. DiALoG will be used in conjunction with "Amplify Science", a Lawrence Hall of Science-developed curriculum that incorporates the science practice of engaging in argument from evidence, and a suite of newly developed Responsive Mini-Lessons (RMLs), which consist of 20-30 minute instructional strategies designed to assist teachers to provide feedback to students' thinking and follow-up to argumentation episodes that the DiALoG tool identifies in need of further support. The study will allow the refinement and expansion of DiALoG and evaluation of its impact on teacher pedagogical content knowledge and formative assessment practices in widespread classroom use.

The project will address two specific research questions: (1) How can DiALoG be refined to provide a formative assessment tool for oral argumentation that is reliable, practical, and useful in middle school classrooms?; and (2) How does the use of DiALoG affect teacher formative assessment practices around evidence-based argumentation, when implementing science units designed to support oral argumentation? In order to answer these questions, the project will conduct a randomized control trial with 100 teachers: 50 will teach argumentation-focused curriculum with DiALoG, 50 will teach the same curriculum without DiALoG. Both control and treatment teachers will receive all digital and physical materials needed to teach three Amplify Science curriculum units. Treatment teachers will be provided also with the most recent version of DiALoG, including the linked RMLs, as well as support materials for using DiALoG with the Amplify curriculum. A subgroup of focus teachers (5 from the treatment group, and 5 from the control group) will be the subject of additional data collection and analysis. Three focus lessons, in which students are engaging in small-group or whole-class oral argumentation, will be selected from each of the three Amplify Science curricular units. Teacher measures for the randomized control trial will include validated instruments, such as (a) a pre- and post-assessment of teacher pedagogical content knowledge; (b) post-lesson and post-unit surveys in which teachers will self-report on their formative assessment practices; and (c) video recordings of selected lessons in the focus classrooms. In order to observe potential differences in formative assessment practices between treatment and control, protocols will be used to analyze the video recordings of focus classrooms, including (a) Reformed Teaching Observation Protocol; (b) Assessment of Scientific Argumentation inside the Classroom; and (c) Formative Assessment for Teachers and Students. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Pages

Subscribe to Mixed Methods