Assessment

Applying Rasch Measurement to Assess Knowledge-in-Use in Science Education

This study applied the many-facet Rasch measurement (MFRM) to assess students’ knowledge-in-use in middle school physical science. 240 students completed three knowledge-in-use classroom assessment tasks on an online platform. We developed transformable scoring rubrics to score students’ responses, including a task-generic polytomous rubric (applicable to the three tasks), a task-specific polytomous rubric (for each task), and a task-specific dichotomous rubric (for each task). Three qualified raters scored 240 students’ responses to the three tasks.

Author/Presenter

Peng He

Xiaoming Zhai

Namsoo Shin

Joseph Krajcik

Lead Organization(s)
Year
2023
Short Description

This study applied the many-facet Rasch measurement (MFRM) to assess students’ knowledge-in-use in middle school physical science.

Reasoning About Data in Elementary School: Student Strategies and Strengths when Reasoning with Multiple Variables

The need for data literacy is an increasingly pressing priority in society, but most of the work in data-centred education has focused on developing skills at the middle school, secondary, and post-secondary levels, with little attention on the potential for engaging elementary-aged students in reasoning with and about data. This paper reports findings from a foundational study to explore the natural strengths, skills, and strategies that upper elementary students bring to reasoning about data-centred problems.

Author/Presenter

Jessica Sickler

Michelle Lentzner

Lynn T. Goldsmith

Lauren Brase

Randall Kochevar

Year
2024
Short Description

The need for data literacy is an increasingly pressing priority in society, but most of the work in data-centred education has focused on developing skills at the middle school, secondary, and post-secondary levels, with little attention on the potential for engaging elementary-aged students in reasoning with and about data. This paper reports findings from a foundational study to explore the natural strengths, skills, and strategies that upper elementary students bring to reasoning about data-centred problems.

Reasoning About Data in Elementary School: Student Strategies and Strengths when Reasoning with Multiple Variables

The need for data literacy is an increasingly pressing priority in society, but most of the work in data-centred education has focused on developing skills at the middle school, secondary, and post-secondary levels, with little attention on the potential for engaging elementary-aged students in reasoning with and about data. This paper reports findings from a foundational study to explore the natural strengths, skills, and strategies that upper elementary students bring to reasoning about data-centred problems.

Author/Presenter

Jessica Sickler

Michelle Lentzner

Lynn T. Goldsmith

Lauren Brase

Randall Kochevar

Year
2024
Short Description

The need for data literacy is an increasingly pressing priority in society, but most of the work in data-centred education has focused on developing skills at the middle school, secondary, and post-secondary levels, with little attention on the potential for engaging elementary-aged students in reasoning with and about data. This paper reports findings from a foundational study to explore the natural strengths, skills, and strategies that upper elementary students bring to reasoning about data-centred problems.

Multidimensional Science Assessment: Design Challenges and Technology Affordances

Contemporary views on what students should learn increasingly emphasize that students need to acquire more than a base of knowledge; they need to acquire the skills and abilities to use such knowledge in dynamic and flexible ways. To be most effective, learning environments need assessments that are aligned to these perspectives. Using a principled design framework can help guide assessment development toward such targets. Even when using a framework, however, thorny design challenges may arise.

Author/Presenter

Brian D. Gane

Diksha Gaur

Samuel Arnold

Daniel Damelin

Lead Organization(s)
Year
2024
Short Description

In this paper, we describe three challenges (conflict between multiple dimensions of science proficiency, authentic data, and grade-appropriate graphing tools) that we faced when designing for a specific Next Generation Science Standard, and the theoretical and design principles that guided us as we ideated design solutions. Through these designs we maintained alignment to our multidimensional assessment targets, a critical component of our larger assessment validity argument.

Multidimensional Science Assessment: Design Challenges and Technology Affordances

Contemporary views on what students should learn increasingly emphasize that students need to acquire more than a base of knowledge; they need to acquire the skills and abilities to use such knowledge in dynamic and flexible ways. To be most effective, learning environments need assessments that are aligned to these perspectives. Using a principled design framework can help guide assessment development toward such targets. Even when using a framework, however, thorny design challenges may arise.

Author/Presenter

Brian D. Gane

Diksha Gaur

Samuel Arnold

Daniel Damelin

Lead Organization(s)
Year
2024
Short Description

In this paper, we describe three challenges (conflict between multiple dimensions of science proficiency, authentic data, and grade-appropriate graphing tools) that we faced when designing for a specific Next Generation Science Standard, and the theoretical and design principles that guided us as we ideated design solutions. Through these designs we maintained alignment to our multidimensional assessment targets, a critical component of our larger assessment validity argument.

Can Generative AI and ChatGPT Outperform Humans on Cognitive-Demanding Problem-Solving Tasks in Science?

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items. Fifty-four 2019 NAEP science assessment tasks were coded by content experts using a two-dimensional cognitive load framework, including task cognitive complexity and dimensionality.

Author/Presenter

Xiaoming Zhai

Matthew Nyaaba

Wenchao Ma

Lead Organization(s)
Year
2024
Short Description

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items.

Can Generative AI and ChatGPT Outperform Humans on Cognitive-Demanding Problem-Solving Tasks in Science?

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items. Fifty-four 2019 NAEP science assessment tasks were coded by content experts using a two-dimensional cognitive load framework, including task cognitive complexity and dimensionality.

Author/Presenter

Xiaoming Zhai

Matthew Nyaaba

Wenchao Ma

Lead Organization(s)
Year
2024
Short Description

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items.

Classroom-Based STEM Assessment: Contemporary Issues and Perspectives

Image
Author/Presenter

Christopher J. Harris, Eric Wiebe, Shuchi Grover, James W. Pellegrino, Eric Banilower, Arthur Baroody, Erin Furtak, Ryan “Seth” Jones, Leanne R. Ketterlin-Geller, Okhee Lee, Xiaoming Zhai

Year
2023
Short Description

This report takes stock of what we currently know as well as what we need to know to make classroom assessment maximally beneficial for the teaching and learning of STEM subject matter in K–12 classrooms.

Myths, Mis- and Preconceptions of Artificial Intelligence: A Review of the Literature

Artificial Intelligence (AI) is prevalent in nearly every aspect of our lives. However, recent studies have found a significant amount of confusion and misunderstanding surrounding AI. To develop effective educational programs in the field of AI, it is vital to examine and understand learners' pre- and misconceptions as well as myths about AI. This study examined a corpus of 591 studies.

Author/Presenter

Arne Bewersdorff

Xiaoming Zhai

Jessica Roberts

Claudia Nerdel

Lead Organization(s)
Year
2023
Short Description

Artificial Intelligence (AI) is prevalent in nearly every aspect of our lives. However, recent studies have found a significant amount of confusion and misunderstanding surrounding AI. To develop effective educational programs in the field of AI, it is vital to examine and understand learners' pre- and misconceptions as well as myths about AI. This study examined a corpus of 591 studies.

ChatGPT for Next Generation Science Learning

This article pilots ChatGPT in tackling the most challenging part of science learning and found it successful in automation of assessment development, grading, learning guidance, and recommendation of learning materials.

Zhai, X. (2023). ChatGPT for Next Generation Science Learning | XRDS: Crossroads, 29(3), 42-46. https://doi.org/10.1145/3589649

Author/Presenter
Xiaoming Zhai
Lead Organization(s)
Year
2023
Short Description

This article pilots ChatGPT in tackling the most challenging part of science learning and found it successful in automation of assessment development, grading, learning guidance, and recommendation of learning materials.