Assessment

Unpacking the Nuances: An Exploratory Multilevel Analysis on the Operationalization of Integrated STEM Education and Student Attitudinal Change

Integrated STEM education (iSTEM) is recognized for its potential to improve students’ scientific and mathematical knowledge, as well as to nurture positive attitudes toward STEM, which are essential for motivating students to consider STEM-related careers. While prior studies have examined the relationship between specific iSTEM activities or curricula and changes in student attitudes, research is lacking on how the aspects of iSTEM are operationalized and their influence on shifts in student attitudes towards STEM, especially when considering the role of demographic factors.

Author/Presenter

Benny Mart R. Hiwatig

Gillian H. Roehrig

Mark D. Rouleau

Lead Organization(s)
Year
2024
Short Description

Integrated STEM education (iSTEM) is recognized for its potential to improve students’ scientific and mathematical knowledge, as well as to nurture positive attitudes toward STEM, which are essential for motivating students to consider STEM-related careers. While prior studies have examined the relationship between specific iSTEM activities or curricula and changes in student attitudes, research is lacking on how the aspects of iSTEM are operationalized and their influence on shifts in student attitudes towards STEM, especially when considering the role of demographic factors. Addressing this gap, our study applied multilevel modeling to analyze how different iSTEM aspects and demographic variables predict changes in student attitudes.

Unpacking the Nuances: An Exploratory Multilevel Analysis on the Operationalization of Integrated STEM Education and Student Attitudinal Change

Integrated STEM education (iSTEM) is recognized for its potential to improve students’ scientific and mathematical knowledge, as well as to nurture positive attitudes toward STEM, which are essential for motivating students to consider STEM-related careers. While prior studies have examined the relationship between specific iSTEM activities or curricula and changes in student attitudes, research is lacking on how the aspects of iSTEM are operationalized and their influence on shifts in student attitudes towards STEM, especially when considering the role of demographic factors.

Author/Presenter

Benny Mart R. Hiwatig

Gillian H. Roehrig

Mark D. Rouleau

Lead Organization(s)
Year
2024
Short Description

Integrated STEM education (iSTEM) is recognized for its potential to improve students’ scientific and mathematical knowledge, as well as to nurture positive attitudes toward STEM, which are essential for motivating students to consider STEM-related careers. While prior studies have examined the relationship between specific iSTEM activities or curricula and changes in student attitudes, research is lacking on how the aspects of iSTEM are operationalized and their influence on shifts in student attitudes towards STEM, especially when considering the role of demographic factors. Addressing this gap, our study applied multilevel modeling to analyze how different iSTEM aspects and demographic variables predict changes in student attitudes.

Teacher Educators’ Use of Formative Feedback During Preservice Teachers’ Simulated Teaching Experiences in Mathematics and Science

The purpose of this research study was to identify how teacher educators (TEs) attend to and use formative feedback as they work to support preservice teachers’ (PSTs’) learning. The formative feedback was provided to the TEs as part of recurring instructional cycles within their elementary mathematics or science methods course. In these instructional cycles, their PSTs prepared for, engaged in, and reflected on their ability to facilitate argumentation-focused discussions in a simulated classroom.

Author/Presenter

Jamie N. Mikeska

Heather Howell

Devon Kinsey

Lead Organization(s)
Year
2024
Short Description

The purpose of this research study was to identify how teacher educators (TEs) attend to and use formative feedback as they work to support preservice teachers’ (PSTs’) learning. The formative feedback was provided to the TEs as part of recurring instructional cycles within their elementary mathematics or science methods course. In these instructional cycles, their PSTs prepared for, engaged in, and reflected on their ability to facilitate argumentation-focused discussions in a simulated classroom. After each cycle, the TEs received formative information about their PSTs’ discussion performance in the form of a feedback report and a scoring report.

Using Artificial Intelligence to Support Peer-to-Peer Discussions in Science Classrooms

In successful peer discussions students respond to each other and benefit from supports that focus discussion on one another’s ideas. We explore using artificial intelligence (AI) to form groups and guide peer discussion for grade 7 students. We use natural language processing (NLP) to identify student ideas in science explanations. The identified ideas, along with Knowledge Integration (KI) pedagogy, informed the design of a question bank to support students during the discussion. We compare groups formed by maximizing the variety of ideas among participants to randomly formed groups.

Author/Presenter

Billings, K., Chang, H-Y., Brietbart, J., & Linn, M.C. 

Short Description

We use natural language processing (NLP) to identify student ideas in science explanations. The identified ideas, along with Knowledge Integration (KI) pedagogy, informed the design of a question bank to support students during the discussion. We compare groups formed by maximizing the variety of ideas among participants to randomly formed groups. 

Using Artificial Intelligence to Support Peer-to-Peer Discussions in Science Classrooms

In successful peer discussions students respond to each other and benefit from supports that focus discussion on one another’s ideas. We explore using artificial intelligence (AI) to form groups and guide peer discussion for grade 7 students. We use natural language processing (NLP) to identify student ideas in science explanations. The identified ideas, along with Knowledge Integration (KI) pedagogy, informed the design of a question bank to support students during the discussion. We compare groups formed by maximizing the variety of ideas among participants to randomly formed groups.

Author/Presenter

Billings, K., Chang, H-Y., Brietbart, J., & Linn, M.C. 

Short Description

We use natural language processing (NLP) to identify student ideas in science explanations. The identified ideas, along with Knowledge Integration (KI) pedagogy, informed the design of a question bank to support students during the discussion. We compare groups formed by maximizing the variety of ideas among participants to randomly formed groups. 

An Empirical Investigation of Neural Methods for Content Scoring of Science Explanations

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out.

Author/Presenter

Riordan, B., Bichler, S., Bradford, A., King Chen, J., Wiley, K., Gerard, L., & Linn, M.C.

Short Description

We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.

An Empirical Investigation of Neural Methods for Content Scoring of Science Explanations

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out.

Author/Presenter

Riordan, B., Bichler, S., Bradford, A., King Chen, J., Wiley, K., Gerard, L., & Linn, M.C.

Short Description

We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.

Productive Problem-Solving Behaviors of Students with Learning Disabilities

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors). Themes emerged from qualitative analysis indicating that the students engaged in problem-solving behaviors, as well as common patterns of behaviors.

Author/Presenter

Emily Evans

Jonathan Bostic

Sean Yee

Lead Organization(s)
Year
2024
Short Description

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors).

Productive Problem-Solving Behaviors of Students with Learning Disabilities

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors). Themes emerged from qualitative analysis indicating that the students engaged in problem-solving behaviors, as well as common patterns of behaviors.

Author/Presenter

Emily Evans

Jonathan Bostic

Sean Yee

Lead Organization(s)
Year
2024
Short Description

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors).

Productive Problem-Solving Behaviors of Students with Learning Disabilities

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors). Themes emerged from qualitative analysis indicating that the students engaged in problem-solving behaviors, as well as common patterns of behaviors.

Author/Presenter

Emily Evans

Jonathan Bostic

Sean Yee

Lead Organization(s)
Year
2024
Short Description

The purpose of this study was to explore the problem-solving behaviors of middle-school students with learning disabilities (SLD). Think-aloud interviews were performed with 20 seventh- and eighth-grade students who had learning disabilities to observe their behaviors while solving mathematical word problems (i.e., behaviors and patterns of behaviors).