Assessment

An Empirical Investigation of Neural Methods for Content Scoring of Science Explanations

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out.

Author/Presenter

Riordan, B., Bichler, S., Bradford, A., King Chen, J., Wiley, K., Gerard, L., & Linn, M.C.

Short Description

We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.

An Empirical Investigation of Neural Methods for Content Scoring of Science Explanations

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out.

Author/Presenter

Riordan, B., Bichler, S., Bradford, A., King Chen, J., Wiley, K., Gerard, L., & Linn, M.C.

Short Description

We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.

Productive Problem-Solving Behaviors of Students with Learning Disabilities

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors). Themes emerged from qualitative analysis indicating that the students engaged in problem-solving behaviors, as well as common patterns of behaviors.

Author/Presenter

Emily Evans

Jonathan Bostic

Sean Yee

Lead Organization(s)
Year
2024
Short Description

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors).

Productive Problem-Solving Behaviors of Students with Learning Disabilities

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors). Themes emerged from qualitative analysis indicating that the students engaged in problem-solving behaviors, as well as common patterns of behaviors.

Author/Presenter

Emily Evans

Jonathan Bostic

Sean Yee

Lead Organization(s)
Year
2024
Short Description

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors).

Productive Problem-Solving Behaviors of Students with Learning Disabilities

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors). Themes emerged from qualitative analysis indicating that the students engaged in problem-solving behaviors, as well as common patterns of behaviors.

Author/Presenter

Emily Evans

Jonathan Bostic

Sean Yee

Lead Organization(s)
Year
2024
Short Description

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors).

Productive Problem-Solving Behaviors of Students with Learning Disabilities

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors). Themes emerged from qualitative analysis indicating that the students engaged in problem-solving behaviors, as well as common patterns of behaviors.

Author/Presenter

Emily Evans

Jonathan Bostic

Sean Yee

Lead Organization(s)
Year
2024
Short Description

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors).

Understanding Variation in Integrated STEM Practice as Measured by the STEM Observation Protocol (STEM-OP)

To better understand integrated STEM education, this work explored scores on the STEM Observation Protocol (STEM-OP), a newly developed observation protocol for use in K-12 science and engineering classrooms. The goals of this work were to better understand how integrated STEM might look throughout an integrated STEM unit and identify limitations of the instrument when examining daily scores and full unit implementation scores. The work takes a mixed methods approach to first examine what scores may be typically seen with daily and unit implementations.

Author/Presenter

Emily A. Dare

Elizabeth A. Ring-Whalen

Year
2024
Short Description

To better understand integrated STEM education, this work explored scores on the STEM Observation Protocol (STEM-OP), a newly developed observation protocol for use in K-12 science and engineering classrooms. The goals of this work were to better understand how integrated STEM might look throughout an integrated STEM unit and identify limitations of the instrument when examining daily scores and full unit implementation scores.

Understanding Variation in Integrated STEM Practice as Measured by the STEM Observation Protocol (STEM-OP)

To better understand integrated STEM education, this work explored scores on the STEM Observation Protocol (STEM-OP), a newly developed observation protocol for use in K-12 science and engineering classrooms. The goals of this work were to better understand how integrated STEM might look throughout an integrated STEM unit and identify limitations of the instrument when examining daily scores and full unit implementation scores. The work takes a mixed methods approach to first examine what scores may be typically seen with daily and unit implementations.

Author/Presenter

Emily A. Dare

Elizabeth A. Ring-Whalen

Year
2024
Short Description

To better understand integrated STEM education, this work explored scores on the STEM Observation Protocol (STEM-OP), a newly developed observation protocol for use in K-12 science and engineering classrooms. The goals of this work were to better understand how integrated STEM might look throughout an integrated STEM unit and identify limitations of the instrument when examining daily scores and full unit implementation scores.

Alliance for Science Educators (ASET) Toolkit

The ASET Toolkit is a Next Generation Science Standards (NGSS) planning toolkit designed to help science educators plan lessons and units that integrate the 3 dimensions outlined in the Framework (NRC, 2012).

Author/Presenter

Michele Korb

Year
2024
Short Description

The ASET Toolkit is a Next Generation Science Standards (NGSS) planning toolkit designed to help science educators plan lessons and units that integrate the 3 dimensions outlined in the Framework (NRC, 2012). The toolkit includes: 1) Dialogic tools that foster and support deepening awareness and understanding of how to make the significant shifts in instruction, learning, and formative assessment required by the new science reform documents, and 2) Analytical tools for revising existing and/or developing new curriculum that better supports 3-dimensional teaching, learning, and formative assessment of student learning.

Next Generation Science Assessment (NGSA) Task Portal

This portal contains classroom-ready assessments for elementary and middle school teachers to use formatively to gain insights into their students’ progress on achieving NGSS performance expectations.

Author/Presenter

James Pellegrino

Carla Strickland

Year
2023
Short Description

This portal contains classroom-ready assessments for elementary and middle school teachers to use formatively to gain insights into their students’ progress on achieving NGSS performance expectations.