Assessment

Improving the Implementation of Rigorous Instructional Materials in Middle-Grades Mathematics: Developing a System of Practical Measures and Routines (Collaborative Research: Jackson)

The goal of this project is to improve the implementation of rigorous instructional materials in middle-grades mathematics at scale through a system of practical measures and routines for collecting and using data that both assesses and supports implementation.

Lead Organization(s): 
Award Number: 
1620851
Funding Period: 
Sat, 10/01/2016 to Thu, 09/30/2021
Full Description: 

The goal of this 5-year research project is to improve the implementation of rigorous instructional materials in middle-grades mathematics at scale. Many projects seek to improve mathematics instruction, but are not able to easily track their efforts at improvement. The primary product of this project will be a system of practical measures and routines for collecting and using data that both assesses and supports the implementation of rigorous instructional materials in middle-grades mathematics. In contrast to research and accountability measures, practical measures are assessments that require little time to administer and can thus be used frequently. The data can be analyzed rapidly so that teachers can receive prompt feedback on their progress, and instructional leaders can use the data to decide where to target resources to support improvement in the quality of instruction and student learning. The system of practical measures and routines will include 1) measures of high-leverage aspects of teachers' instructional practices that have been linked to student learning (e.g., rigor of tasks, quality of students' discourse) and attend to equitable student participation; and 2) measures of high-leverage aspects of key supports for improving the quality of teachers' practice (e.g., quality of professional development; coaching); and 3) a set of routines regarding how to use the resulting data to engage in rapid, improvement efforts. A key principle of the proposed project is that the system of measures and routines can be adapted to a wide range of school and district contexts. This project is supported by the Discovery Research preK-12 (DRK-12) program. The DRK-12 program supports research and development of STEM education innovations and approaches in assessment, learning, and teaching.

The project will establish three research-practice partnerships with five districts, in three different states, that are currently implementing rigorous instructional materials in middle-grades mathematics. Year 1 will focus on the development of a set of practical measures of classroom instruction. Year 2 will focus on testing the use of the classroom measures in the context of supports for teachers' learning, and the development of practical measures of key supports for teachers' learning. Years 3-4 will focus on how the project can "learn our way to scale" (Bryk et al., 2015), which requires strategically implementing measures and routines in increasingly diverse conditions. The project will engage in rapid improvement cycles in which researchers will work alongside district leaders and professional development (PD) facilitators to analyze the data from the measures of both classroom instruction and the quality of support for teacher learning to test the effectiveness of improvements in intended supports for teacher learning and to adjust the design of the support based on data. Across Years 1-4, the project will use recent developments in technology and information visualization to test and improve 1) the collection of practical measures in situ and 2) the design of data representations (or visualizations) that support teachers and leaders to make instructional improvement decisions. In Year 5, the project will conduct formal analyses of the relations between supports for teachers' learning; teachers' knowledge and classroom practices; and student learning.

Connected Biology: Three-Dimensional Learning from Molecules to Populations (Collaborative Research: White)

This project will design, develop, and examine the learning outcomes of a new curriculum unit for biology that embodies the conceptual framework of the Next Generation Science Standards (NGSS). The curriculum materials to be developed by this project will focus on two areas of study that are central to the life sciences: genetics and the processes of evolution by natural selection.

Lead Organization(s): 
Award Number: 
1620746
Funding Period: 
Sat, 10/01/2016 to Wed, 09/30/2020
Full Description: 

This project will contribute to this mission by designing, developing, and examining the learning outcomes of a new curriculum unit for biology that embodies the conceptual framework of the Next Generation Science Standards (NGSS). The curriculum materials to be developed by this project will focus on two areas of study that are central to the life sciences: genetics and the processes of evolution by natural selection. These traditionally separate topics will be interlinked and will be designed to engage students in the disciplinary core ideas, crosscutting concepts, and the science and engineering practices defined by the NGSS. Once developed, the curriculum materials will be available online for use in high school biology courses nationwide.

This project will be guided by two main research questions: 1) How does learning progress when students experience a set of coherent biology learning materials that employ the principles of three-dimensional learning?; and 2) How do students' abilities to transfer understanding about the relationships between molecules, cells, organisms, and evolution change over time and from one biological phenomenon to another? The project will follow an iterative development plan involving cycles of designing, developing, testing and refining elements of the new curricular model. The project team will work with master teachers to design learning sequences that use six case studies to provide examples of how genetic and evolutionary processes are interlinked. An online data exploration environment will extend learning by enabling students to simulate phenomena being studied and explore data from multiple experimental trials as they seek patterns and construct cause-and-effect explanations of phenomena. Student learning will be measured using a variety of assessment tools, including multiple-choice assessment of student understanding, surveys, classroom observations and interviews, and embedded assessments and log files from the online learning environment.

Developing A Discourse Observation Tool and Online Professional Development to Promote Science, Oral Language and Literacy Development from the Start of School

The goal of this project is to develop a classroom observation tool and an online professional development model to help early-elementary teachers improve science instruction among young learners by cultivating scientific discourse.

Lead Organization(s): 
Award Number: 
1620580
Funding Period: 
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description: 

The goal of this project is to develop resources and a professional development model to help early-elementary teachers improve science instruction among young learners by cultivating scientific discourse. A central component of the Next Generation Science Standards (NGSS) is engaging students in discourse with a focus on formulating and communicating scientific explanations. This project will develop a classroom observation tool that will help teachers examine changes in the quantity and quality of science discourse in K-2 classrooms over time. The project will also develop an online professional development (PD) model that uses the new observation tool to help teachers analyze their own classroom practices and the practice of others to improve classroom efforts to foster improved scientific discourse.

This early stage design and development study will employ a Design-Based Implementation Research (DBIR) approach to develop the new classroom observation tool and online professional development model, and then seek answers to the following research questions: 1) How can a classroom observation measure be developed to effectively capture the range in quality of science discourse in early elementary classrooms?; 2) How can an online PD model be developed based on the new observation tool?; 3) How do teachers' knowledge and instructional practice change over the course of participation in the yearlong PD?; and 4) How does the quantity and quality of science discourse change in K-2 classrooms over the course of teachers' participation in a yearlong online PD experience that is built around the new observation tool? The project will engage 36 teachers and their 36 different classrooms in Michigan and use multiple data sources to understand whether and how teacher knowledge and instructional practices change during participation in the new PD model. Multiple iterations of design, data collection, and refinement will be used to understand how, when, and why features of the PD and observation tool might combine to transform science discourse in early elementary classrooms. In years 3 and 4, the project team will conduct two year-long implementation trials with cohorts of 15 teachers and 5 instructional coaches (experienced science teachers) who will use the PD and tool in order study their implementation and make iterative improvements. The project will also gather data to understand changes in teacher knowledge and practice as well as video data to document changes in classroom discourse.

Misconceptions Oriented Standards-Based Assessment Resource for Teachers of High School Physical Sciences (MOSART HSPS)

This project builds upon the widely used K-12 Misconception Oriented Standards-based Assessment Resource for Teachers (MOSART). The project is developing 500 new test items that are intended to assess disciplinary core ideas in chemistry and physics aligned to Next Generation Science Standards. The new measures will be used to measure the knowledge acquired in a year of study by 10,000 students and 200 teachers in chemistry and physics.

Lead Organization(s): 
Award Number: 
1621210
Funding Period: 
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description: 

The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

Researchers in the Harvard Smithsonian Center for Astrophysics at Harvard University are developing and validating assessment instruments intended to measure chemistry and physical science concepts for students and teachers in grades 9 through 12. This project builds upon the widely used K-12 Misconception Oriented Standards-based Assessment Resource for Teachers (MOSART) developed by this research team. The project is developing 500 new test items that are intended to assess disciplinary core ideas in chemistry and physics aligned to Next Generation Science Standards. The new measures will be used to measure the knowledge acquired in a year of study by 10,000 students and 200 teachers in chemistry and physics. The new assessment items and instruments will be made available to other researchers and practitioners through the project website and the on-line MOSART assessment system.

The assessment development process is based on prior research conducted to develop similar MOSART items and instruments, which includes design efforts of assessment specialists, content experts, and research scientists. Pilot items are tested with a national sample of approximately 20,000 high school students and their teachers. Data will be analyzed using item response theory to model student responses. Outcomes consist of item parameters, test and sub-test characteristics, and predictive linkages among items. Descriptive statistics are generated to establish the state of student knowledge, pre-and post-test performance by item and by standard, and teacher knowledge. Descriptive analyses are followed by hierarchical linear modeling (HLM) to examine the relationships between teacher-level and program-level variables.

The MOSART instruments have been widely used and are based on a model of cognition with a strong research base in misconceptions in science education. These additional Grade 9-12 chemistry and physics instruments will address gaps in the current MOSART system of assessments. The new instruments focused on chemistry and physics disciplinary core ideas provide a much needed set of assessments for researchers and practitioners, particularly teacher professional development providers.

Developing Preservice Elementary Teachers' Ability to Facilitate Goal-Oriented Discussions in Science and Mathematics via the Use of Simulated Classroom Interactions

The project will develop, pilot, and validate eight discussion-oriented performance tasks that will be embedded in an online simulated classroom environment. The resulting research and development products could be used nationwide in teacher preparation and professional development settings to assess and develop teachers' ability to support classroom discussion in science and mathematics.

Lead Organization(s): 
Award Number: 
1621344
Funding Period: 
Mon, 08/01/2016 to Fri, 07/31/2020
Full Description: 

There is widespread recognition in educational literatures that academic discourse is important for supporting students' developing understanding in the disciplines of science and mathematics. College and career-ready standards also call for attention to supporting students' learning of how to think and communicate like disciplinary experts. The teaching practice of orchestrating classroom discussion is intended to support students in obtaining higher levels of academic achievement but also to support students' participation in a democratic society. However, research has found that teachers--particularly novice teachers--struggle to orchestrate discussion effectively for science and mathematics. The investigators of this project hypothesize that opportunities to 1) practice orchestrating discussions in simulated classroom environments; 2) receive constructive feedback on their practice; and 3) reflect on that feedback and their experiences with peers and teacher educators, develops preservice teachers' abilities to lead productive classroom discussion. This may allow them to be more effective at orchestrating discussion when they begin teaching real students in science and mathematics classrooms. The project team, which includes investigators from Educational Testing Service (ETS) and software engineers at Mursion, will develop, pilot, and validate eight discussion-oriented performance tasks that will be embedded in an online simulated classroom environment. The resulting research and development products could be used nationwide in teacher preparation and professional development settings to assess and develop teachers' ability to support classroom discussion in science and mathematics.

The Discovery Research K-12 (DRK-12) program seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models, and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This Early Stage Design and Development project will 1) iteratively develop, pilot, and refine eight science and mathematics discussion-oriented performance tasks (six formative, two summative), scoring rubrics, and rater training materials; 2) deploy the intervention in four university sites, collecting data from 240 prospective teachers in both treatment and business-as-usual courses; and 3) use data analyses and expert review to build a five-part argument for the validity of the assessment and scoring rubrics. Data sources include prospective teachers' background and demographic information, cognitive interviews, surveys, scores on content knowledge for teaching (CKT) instruments, performance and scores on the developed performance tasks, discussion scores on Danielson's Framework for Teaching observation protocol, and case study interviews with prospective teachers. The project team will also conduct interviews with teacher educators and observe classroom debrief sessions with prospective teachers and their teacher educators. The research will examine each teacher's scores on two summative performance tasks administered pre- and post-intervention and will look for evidence of growth across three formative tasks. Linear regression models will be used to understand relationships among teachers' CKT scores, pre-intervention performance task scores, group assignment, and post-intervention performance task scores. A grounded theory approach to coding qualitative data of 24 case study teachers, observations of debrief sessions, and interviews with teacher educators will generate descriptive use cases, illustrating how the tools can support prospective teachers in learning how to facilitate discussions focused on science and mathematics argumentation. Mursion will develop a webpage on its website dedicated to this project that will allow the team to post the new performance-based tasks, scoring rubrics, and examples of performance in the simulated environment for teacher educators, educational researchers, and policy makers and collect feedback from them that can be used as another information source for refining tools and their use. Research findings will also be disseminated by more traditional means, such as papers in peer-reviewed research and practitioner journals and conference presentations.


Project Videos

2019 STEM for All Video Showcase

Title: Simulated Classrooms as Practice-Based Learning Spaces

Presenter(s): Jamie Mikeska, Heather Howell, & Carrie Straub

2018 STEM for All Video Showcase

Title: Leading Science/Math Discussions in a Simulated Classroom

Presenter(s): Heather Howell, Jamie Mikeska, & Carrie Straub

 2017 STEM for All Video Showcase
Title: Simulated Classroom Environments for Discussions

Presenter(s): Jamie Mikeska, Heather Howell, & Carrie Straub


CAREER: Multilevel Mediation Models to Study the Impact of Teacher Development on Student Achievement in Mathematics

This project will develop a comprehensive framework to inform and guide the analytic design of teacher professional development studies in mathematics. An essential goal of the research is to advance a science of teaching and learning in ways that traverse both research and education.

Lead Organization(s): 
Award Number: 
1552535
Funding Period: 
Thu, 09/01/2016 to Tue, 08/31/2021
Full Description: 

This is a Faculty Early Career Development Program (CAREER) project. The CAREER program is a National Science Foundation-wide activity that offers the most prestigious awards in support of junior faculty who exemplify the role of teacher-scholars through outstanding research, excellent education and the integration of education and research. The intellectual merit and broader impacts of this study lie in two complementary contributions of the project. First, the development of the statistical framework for the design of multilevel mediation studies has significant potential for broad impact because it develops a core platform that is transferable to other STEM (science, technology, engineering, and mathematics) education areas and STEM disciplines. Second, the development of software and curricular materials to implement this framework further capitalize on the promise of this work because it distributes the results in an accessible manner to diverse sets of research and practitioner groups across STEM education areas and STEM disciplines. Together, the components of this project will substantially expand the scope and quality of evidence generated through mathematics professional development and, more generally, multilevel mediation studies throughout STEM areas by increasing researchers' capacity to design valid and comprehensive studies of the theories of action and change that underlie research programs.

This project will develop a comprehensive framework to inform and guide the analytic design of teacher professional development studies in mathematics. The proposed framework incorporates four integrated research and education components: (1) develop statistical formulas and tools to guide the optimal design of experimental and non-experimental multilevel mediation studies in the presence of measurement error, (2) develop empirical estimates of the parameters needed to implement these formulas to design teacher development studies in mathematics, (3) develop free and accessible software to execute this framework, and (4) develop training materials and conduct workshops on the framework to improve the capacity of the field to design effective and efficient studies of teacher development. An essential goal of the research is to advance a science of teaching and learning in ways that traverse both research and education.

Science, Technology, Engineering and Mathematics Scholars Teacher Academy Resident System

This project will investigate the effectiveness of a teacher academy resident model to recruit, license, induct, employ, and retain middle school and secondary teachers for high-need schools in the South. It will prepare new, highly-qualified science and mathematics teachers from historically Black universities in high-needs urban and rural schools with the goal of increasing teacher retention and diversity rates.

Lead Organization(s): 
Award Number: 
1621325
Funding Period: 
Fri, 07/15/2016 to Wed, 06/30/2021
Full Description: 

This project at Jackson State University will investigate the effectiveness of a teacher academy resident model to recruit, license, induct, employ, and retain middle school and secondary science and mathematics teachers for high-need schools in the South. It will prepare new, highly-qualified science and mathematics teachers from historically Black universities in high-needs urban and rural schools. The project involves a partnership among three historically Black universities (Jackson, State University, Xavier University of Louisiana, and the University of Arkansas at Pine Bluff), and diverse urban and rural school districts in Jackson, Mississippi; New Orleans, Louisiana; and Pine Bluff Arkansas region that serve more than 175,000 students.

Participants will include 150 middle and secondary school teacher residents who will gain clinical mentored experience and develop familiarity with local schools. The 150 teacher residents supported by the program to National Board certification will obtain: state licensure/certification in science teaching, a master's degree, and initiation. The goal is to increase teacher retention and diversity rates. The research question guiding this focus is: Will training STEM graduates have a significant effect on the quality of K-12 instruction, teacher efficacy and satisfaction, STEM teacher retention, and students? Science and mathematics achievement? A quasi-experimental design will be used to evaluate project's effectiveness.

Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum (Collaborative Research: Greenwald)

The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Partner Organization(s): 
Award Number: 
1621441
Funding Period: 
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description: 

This is an early-stage design and development collaborative study submitted to the assessment strand of the Discovery Research PreK-12 (DRK-12) program, in response to Program Solicitation NSF 15-592. The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. To achieve this purpose, the project will examine the validity of a new technology-based formative assessment tool for classroom argumentation--"Diagnosing the Argumentation Levels of Groups" (DiALoG)--for which psychometric validation work has been conducted in a laboratory setting. The DiALoG assessment tool allows teachers to document classroom talk and display scores across multiple dimensions--both intrapersonal and interpersonal--for formative assessment purposes. The project will work with 6th-8th grade science teachers to monitor and support argumentation through real-time formative assessment data generated by the DiALoG instrument. DiALoG will be used in conjunction with "Amplify Science", a Lawrence Hall of Science-developed curriculum that incorporates the science practice of engaging in argument from evidence, and a suite of newly developed Responsive Mini-Lessons (RMLs), which consist of 20-30 minute instructional strategies designed to assist teachers to provide feedback to students' thinking and follow-up to argumentation episodes that the DiALoG tool identifies in need of further support. The study will allow the refinement and expansion of DiALoG and evaluation of its impact on teacher pedagogical content knowledge and formative assessment practices in widespread classroom use.

The project will address two specific research questions: (1) How can DiALoG be refined to provide a formative assessment tool for oral argumentation that is reliable, practical, and useful in middle school classrooms?; and (2) How does the use of DiALoG affect teacher formative assessment practices around evidence-based argumentation, when implementing science units designed to support oral argumentation? In order to answer these questions, the project will conduct a randomized control trial with 100 teachers: 50 will teach argumentation-focused curriculum with DiALoG, 50 will teach the same curriculum without DiALoG. Both control and treatment teachers will receive all digital and physical materials needed to teach three Amplify Science curriculum units. Treatment teachers will be provided also with the most recent version of DiALoG, including the linked RMLs, as well as support materials for using DiALoG with the Amplify curriculum. A subgroup of focus teachers (5 from the treatment group, and 5 from the control group) will be the subject of additional data collection and analysis. Three focus lessons, in which students are engaging in small-group or whole-class oral argumentation, will be selected from each of the three Amplify Science curricular units. Teacher measures for the randomized control trial will include validated instruments, such as (a) a pre- and post-assessment of teacher pedagogical content knowledge; (b) post-lesson and post-unit surveys in which teachers will self-report on their formative assessment practices; and (c) video recordings of selected lessons in the focus classrooms. In order to observe potential differences in formative assessment practices between treatment and control, protocols will be used to analyze the video recordings of focus classrooms, including (a) Reformed Teaching Observation Protocol; (b) Assessment of Scientific Argumentation inside the Classroom; and (c) Formative Assessment for Teachers and Students. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum (Collaborative Research: Henderson)

The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Lead Organization(s): 
Award Number: 
1621496
Funding Period: 
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description: 

This is an early-stage design and development collaborative study submitted to the assessment strand of the Discovery Research PreK-12 (DRK-12) program, in response to Program Solicitation NSF 15-592. The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. To achieve this purpose, the project will examine the validity of a new technology-based formative assessment tool for classroom argumentation--"Diagnosing the Argumentation Levels of Groups" (DiALoG)--for which psychometric validation work has been conducted in a laboratory setting. The DiALoG assessment tool allows teachers to document classroom talk and display scores across multiple dimensions--both intrapersonal and interpersonal--for formative assessment purposes. The project will work with 6th-8th grade science teachers to monitor and support argumentation through real-time formative assessment data generated by the DiALoG instrument. DiALoG will be used in conjunction with "Amplify Science", a Lawrence Hall of Science-developed curriculum that incorporates the science practice of engaging in argument from evidence, and a suite of newly developed Responsive Mini-Lessons (RMLs), which consist of 20-30 minute instructional strategies designed to assist teachers to provide feedback to students' thinking and follow-up to argumentation episodes that the DiALoG tool identifies in need of further support. The study will allow the refinement and expansion of DiALoG and evaluation of its impact on teacher pedagogical content knowledge and formative assessment practices in widespread classroom use.

The project will address two specific research questions: (1) How can DiALoG be refined to provide a formative assessment tool for oral argumentation that is reliable, practical, and useful in middle school classrooms?; and (2) How does the use of DiALoG affect teacher formative assessment practices around evidence-based argumentation, when implementing science units designed to support oral argumentation? In order to answer these questions, the project will conduct a randomized control trial with 100 teachers: 50 will teach argumentation-focused curriculum with DiALoG, 50 will teach the same curriculum without DiALoG. Both control and treatment teachers will receive all digital and physical materials needed to teach three Amplify Science curriculum units. Treatment teachers will be provided also with the most recent version of DiALoG, including the linked RMLs, as well as support materials for using DiALoG with the Amplify curriculum. A subgroup of focus teachers (5 from the treatment group, and 5 from the control group) will be the subject of additional data collection and analysis. Three focus lessons, in which students are engaging in small-group or whole-class oral argumentation, will be selected from each of the three Amplify Science curricular units. Teacher measures for the randomized control trial will include validated instruments, such as (a) a pre- and post-assessment of teacher pedagogical content knowledge; (b) post-lesson and post-unit surveys in which teachers will self-report on their formative assessment practices; and (c) video recordings of selected lessons in the focus classrooms. In order to observe potential differences in formative assessment practices between treatment and control, protocols will be used to analyze the video recordings of focus classrooms, including (a) Reformed Teaching Observation Protocol; (b) Assessment of Scientific Argumentation inside the Classroom; and (c) Formative Assessment for Teachers and Students. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Supporting Chemistry Teachers to Assess and Foster Chemical Thinking

The fundamental purpose of this project is to develop, implement, and study a professional development (PD) model for improving chemistry teachers' formative assessment practices to foster teaching focused on chemical thinking.

Award Number: 
1621228
Funding Period: 
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description: 

This is a design and development study submitted to the teaching strand of the Discovery Research PreK-12 (DRK-12) program; responsive to Program Solicitation NSF 15-592. The DRK-12 program seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by PreK-12 students and teachers, through research and development of STEM education innovations and approaches. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

The fundamental purpose of this project is to develop, implement, and study a professional development (PD) model for improving chemistry teachers' formative assessment practices to foster teaching focused on chemical thinking. The PD model seeks to refocus and enhance teachers' abilities to notice, interpret, and respond to students' ideas. Building on previous exploratory work through which a Chemical Thinking Framework was developed, the proposed effort will work with 8th-12th grade teachers in Boston Public Schools and the New England Region to assist them (a) to recognize tools that are useful in eliciting students' chemical thinking, and adapt or design formative assessments; (b) to make sense of students' chemical thinking based on data collected using formative assessments that elicit students' thinking; and (c) to strategize responsive actions that better foster learning chemistry. The research questions will be: (1) How does chemistry teachers' assessment reasoning change through engagement in PD that focuses on formative assessment as a transformative lever?; and (2) How does engagement in the proposed PD activities influence the ideas and practices that teachers emphasize in their classrooms?

In order to address the research questions, the project will develop a yearlong PD model with four cohorts of 8th-12th grade teachers, including one cohort with teachers from the New England region in a hybrid format (face-to-face and online); each having six teachers (N=24). The model development will be conducted in three phases. In Phase 1, the research team will develop a detailed plan for the PD program by designing and testing conceptualized activities. During Phase 2, the project will study the model with Cohorts 1 and 2 teachers. Phase 3 will focus on positioning the model for scaling up purposes with Cohorts 3 and 4. This phase will test the resources developed, and make comparisons to assess the scalability of the model. Data gathering strategies will include: (a) focus groups to collect data on teachers' assessment reasoning while collectively analyzing students' written work and videos of assessment practice; (b) assessment portfolios to gather individual data on teacher assessment reasoning and practice; (c) assessment snapshots to capture individual teachers' interactions with students; and (d) follow-up sessions to observe and videotape teachers during the year. Data interpretation strategies will include: (a) analysis of domain-neutral factors to characterize changes in how teachers frame and approach assessment of student understanding; and (b) analysis of domain-dependent factors to characterize changes in teachers' attention to the disciplinary ideas of students' work according to the Chemical Thinking Framework. The project will include an external evaluator to address both formative and summative components of this process. The outcome of the proposed scope of work will be a research-informed and field-tested PD model focused on the use of formative assessment to improve chemistry teaching and learning.

Pages

Subscribe to Assessment