Physical Sciences: Chemistry & Physics Education

Close up image of bubblesThis Spotlight highlights seven DRK-12 projects that are advancing the field of physical science education in Chemistry and Physics. They share information on how they are innovating approaches to teaching and learning or assessing concepts and practices in these disciplines; how they are leveraging scientific and technological advances; challenges that are unique to work in this discipline along with strategies for addressing them; and how they are supporting equitable teaching and learning. In a blog post, Ramon E. Lopez also discusses the future of physics and the opportunity and challenge associated with translating advances in the discipline to PreK-12 classrooms.

In this Spotlight:


A Quantum Leap: Teaching & Learning the Physics of the Future

Ramon E. Lopez, Distinguished Professor of Physics, The University of Texas at Arlington

Headshot of Ramon E. LopezTranslating the latest science into education is always a challenge, particularly in physics. A key aspect of exemplary science education is that the students are doing science, not just reading or hearing about science. Physics ideas are often complex and far removed from everyday experience, making it difficult to turn developments in physics into instructional materials appropriate for the classroom. It takes a team of people with deep scientific and pedagogical content knowledge to develop such materials, and classroom testing with students to determine if the instructional materials are effective and practical.

One of the most important advancements over the past hundred years has been the development of quantum mechanics. It is difficult to overstate the impact that this has had on human technology and civilization. All of our modern electronics, our understanding of how stars work, and lasers with their myriad applications are just some of the things that depend on our knowledge of quantum mechanics. Moreover, we are in the midst of another quantum-based technological revolution, with quantum cryptography and quantum computing leading the way. As Yogi Berra put it, “The future ain’t what it used to be.”...Read more.


Featured Projects

Crowd-Sourced Online Nexus for Developing Assessments of Middle-School Physical Science Disciplinary Core Ideas

PI: Philip Sadler | Co-PIs: Gerhard Sonnert, Susan Sunbury
Disciplines: Physical Science
Grade Levels: Middle School

Project Description: The goal of this research project is to determine the extent to which teachers can improve items previously generated by content experts that have been identified to have significant weaknesses (e.g., being too difficult or too easy, being unable to discriminate between high- and low-knowledge students, showing differential performance between students of different genders or racial/ethnic groups). We selected middle school physical science assessment items that needed revision across all four NGSS DCI standards--Matter and its Interactions; Motion and Stability: Forces and Interactions; Energy; and Waves and their Applications in Technologies for Information Transfer--from our Misconceptions-Oriented Standards-Based Assessment Resources for Teachers (MOSART) item bank. Each item was written by an expert and needed improvement either in difficulty, discrimination, or gender or racial bias. Teachers tried to rewrite these items with the goal to improve upon them. Original items and revised items were pilot tested using Amazon Mechanical Turk (AMT). Data were collected from close to 150 subjects for each item. Results for each item tested on AMT were then analyzed using Classical Test Theory to determine psychometric profiles. All revised items with improved characteristics, their matching original item, and one matching item whose characteristics did not improve were included in a field test of middle school physical science students. Field test results were analyzed using Item Response Theory. From these data we found that teachers succeeded in improving item difficulty, discrimination, and in reducing gender bias. However, we observed that teachers were less successful in reducing racial bias.

Practices Supported: The project gauges the merits of a novel collaborative system for the development and validation of high-quality physical science assessment items. It builds on earlier research showing that a crowd-sourced sample works well as an initial proxy for grade 6-8 science students, allowing for rapid feedback on item quality (often in a few days), with Classical Test Theory and Item Response Theory used to establish difficulty, discrimination, and differential item functioning (gender and racial/ethnicity bias). The results of this project are beneficial to middle school physical science teachers. Those revising items have learned how their revisions fared, and those attending professional development sessions about the project have learned how psychometrics are used in test development and how to write meaningful multiple-choice physical science assessment items. Educators also have access to valid and reliable instruments to measure understanding and gains in their own classrooms and to a large item bank of proven items. 

Initial Findings: Analysis indicated that teachers were able to revise questions improving difficulty, discrimination and gender bias. 

Example of original test item and teacher-modified test items.
Example of original test item and teacher-modified test items.

 

  • Improvement on Item Difficulty 
    The goal was to examine whether teachers were able to make items easier. For this portion of the analysis, we focused on items that exhibited difficulty less than 0.5 (i.e., less than 50% of the respondents answered the question correctly). Our analytical sample thus consisted of 14 reference items in the experimental group (i.e., teachers were instructed to make the items easier) and 14 reference items in the control group (i.e., teachers were instructed to improve the items in ways other than difficulty, such as reducing gender bias). As such, there was a total of 39 teacher-revised items in the experimental group and 30 teacher-revised items in the control group. In Figure 1, we plotted the difficulty indices of the original items against the teacher revised ones across groups. If the revised items have the same difficulty as the original ones, they will fall along the diagonal line crossing the origin. If the revised items are easier, they will fall above the diagonal line; otherwise, they will fall below the diagonal line. Since item difficulty represents the proportion of correct answers, the larger the value the easier the item. As shown in Figure 1, 27 out of 39 (69%) revised items fell above the diagonal line, indicating that most of the teachers were able to improve item difficulty, especially for the difficult ones (i.e., black boxes clustering near the origin). We also fit two regression models to each respective group and displayed the best fitting lines. The best fitting line in the experimental group fell above the diagonal line, whereas the best fitting line for the control group did not. Altogether, results indicated that teachers were largely successful in making items easier, especially for the more difficult ones.

Difficulty indices of the original items and the teacher revised items
Figure 1. Difficulty indices of the original items and the teacher revised items. Notes: Black boxes are the data points representing teachers’ attempts to improve item difficulty (i.e., making items easier). Blue triangles are the data points representing teachers’ attempts to improve the items in ways other than difficulty. Item difficulty ranges from 0 to 1. Items with larger values are easier.

 

  • Improvement on Item Discrimination
    The goal was to examine whether teachers were able to make items more discriminating (i.e., items that can better discriminate between students who do well on the overall test and those who do poorly). For this portion of the analysis, we focused on 13 reference items that demonstrated low discrimination (i.e., ≤0.3). Our analytical sample thus consisted of 5 items in the experimental group (i.e., teachers were instructed to make the items more discriminating) and 8 items in the control group (i.e., teachers were instructed to improve the items in ways other than discrimination, such as improving difficulty). As such, there was a total of 13 teacher revised items in the experimental group and 21 teacher revised items in the control group. Figure 2 visually displays the discrimination indices of the original items against those of the teacher revised ones across groups. As with difficulty, if the revised items have the same discrimination as the original ones, they will fall along the diagonal line crossing the origin. If the revised items are more discriminating, they will fall above the diagonal line; otherwise, they will fall below the diagonal line. Results suggest that most teachers were able to improve discrimination, as shown in Figure 2, because nine out of 13 (69%) revised items (i.e., black boxes) fell above the diagonal line. Further, we observed that the best fitting line also fell above the diagonal line, providing additional evidence of teachers’ successful attempts at improving discrimination. We also noted that, although teachers in the control group were instructed to improve items other than discrimination, they nevertheless were able to improve discrimination, but somewhat less successfully (i.e., most of the blue triangles hovered around the diagonal line, indicating positive but small improvement). 

Discrimination indices of the original items and the teacher revised items
Figure 2. Discrimination indices of the original items and the teacher revised items. Notes: Black boxes are the data points representing teachers’ attempts to improve item discrimination. Blue triangles are the data points representing teachers’ attempts to improve the items in ways other than discrimination. Item discrimination ranges from -1 to 1. Items with larger values are more discriminating. 

 

  • Reducing Gender Bias
    For this portion of the analysis, we focused on five reference items that were biased against females (i.e., more difficult for females) as the goal was to examine whether teachers could reduce the gap in difficulty between males and females. There was a total of 16 teacher revised items for those five reference items and 72 revised items for 29 reference items in the control group where the instruction was different from gender bias reduction. In Figure 3 we plotted the difference scores between males and females for the original items against the difference scores for the teacher revised items across groups. If the revised items fall on the diagonal line, it means that there is no improvement or that the bias remains the same. Revised items that fall above the diagonal line are more biased against females whereas the ones that fall below the diagonal line are less biased against females. As shown, 10 out of 16 (63%) revised items (i.e., black boxes) fell below the diagonal line and the best fitting line was pulled down towards the horizontal axis (albeit crossing the diagonal line), indicating that teachers were largely successful in reducing gender bias, i.e., making the items less biased against females. 

Difference in difficulty between males and females of the original items and the teacher revised items
Figure 3. Difference in difficulty between males and females of the original items and the teacher revised items. Notes: Black boxes are the difference scores in difficulty between males and females representing teachers’ attempts to reduce gender bias. Blue triangles are the difference scores in difficulty between males and females representing teachers’ attempts to improve the items in ways other than reducing gender bias. Larger values indicate greater bias against females.

 

  • Reducing Bias Against Under-Represented Minorities (URMs)
    For this portion of the analysis, we included all reference items that were biased against underrepresented racial minorities composed of Black and Hispanic students. As with gender bias, the goal was to examine whether teachers could reduce the gap in difficulty between non-URMs and URMs. There was a total of 35 teacher revised items for 15 reference items in the experimental group and 53 revised items for 19 reference items in the control group where the instruction was different from racial bias reduction. In figure 4 we plotted the difference scores between non-URMs and URMs for the original items against the difference scores for the teacher revised items across groups. Again, if the revised items fall on the diagonal line, it means that there is no improvement or that the bias remains the same. Revised items that fall above the diagonal line are more biased against URMs whereas the ones that fall below the diagonal line are less biased against URMs. As shown, 19 out of 35 (54%) of the revised items (i.e., black boxes) fell above the diagonal line, indicating that teachers were less successful in reducing racial bias, i.e., making the items less biased towards URMs. 

Difference in difficulty between non-URMs and URMs of the original items and the teacher revised items
Figure 4. Difference in difficulty between non-URMs and URMs of the original items and the teacher revised items. Notes: Black boxes are the difference scores in difficulty between non-URMs and URMs representing teachers’ attempts to reduce racial bias. Blue triangles are the difference scores in difficulty between non-URMs and URMs representing teachers’ attempts to improve the items in ways other than reducing racial bias. Larger values indicate greater bias against URMs.

 

Products: The project will produce two assessment instruments (25 questions each) for educators. The first test will include items representing the middle school NGSS physical science standard PS1: Matter and its Interactions, and the second test will include items for physical science standards PS2: Motion and Stability: Forces and Interactions, PS3: Energy, and PS4: Waves and Their Applications in Technologies for Information Transfer. Each test will include items with a range of difficulty, discrimination, and misconception strength. The items will also be selected to minimize biases against groups underrepresented in science (e.g., females, ethnic and racial minorities). These tests, each with an accompanying document that includes an answer key, NGSS DCIs covered, and misconceptions probed, will be posted to the MOSART Self-Service website as a free resource for educators and researchers.


VisChem Logo

Design Research on the Teaching and Learning of Conceptual Understanding in High School Chemistry Though the Use of Dynamic Visualizations of Physical and Chemical Changes

PI: Ellen Yezierski
Disciplines: NGSS Disciplinary Core Ideas (Matter & its Interactions, Motion & Stability: Forces & Interactions, Energy) as found in secondary chemistry
Grade Levels: 9-12

Project Description: This project has responded to a critical need to transform chemistry teaching and learning from an emphasis on description of phenomena to deep understanding consistent with the Next Generation Science Standards (NGSS). The project delivered VisChem Institutes (two remote and one in-person intensive VisChem Institutes at Miami University in Oxford, Ohio) in which secondary chemistry teachers learned the VisChem Approach. The approach uses carefully produced dynamic visualizations with teaching strategies informed by a cognitive learning model. Key to VisChem is communication of internal visualizations using storyboards (drawings with explanation) of chemical and physical changes. The research outcomes from the project (9 peer-reviewed publications) describe and explain how teachers’ use of dynamic, molecular-level visualizations in secondary chemistry classrooms afford and constrain student learning as well as inform students’ ideas about molecular-level behavior underpinning chemical and physical changes. VisChem has established a sustained community of practice of secondary chemistry teachers skilled in the VisChem Approach as well as a group of new teaching and research scholars with expertise in building conceptual understanding through the effective use of visualization. The project will continue in 2025 by remotely delivering more VisChem Institutes, growing the VisChem community of practice, and disseminating VisChem animations, storyboard templates, teacher resources, and assessments through VisChem.org.

Practices Supported: The VisChem approach improves the two Science and Engineering Practices: (1) developing & using models; and (2) constructing explanations.

Challenges in Physical Sciences Education & Strategies for Addressing Them: The overuse of symbols in chemistry--without exploring and making sense of the phenomena that teachers visualize when they use such symbols--is one ubiquitous barrier to conceptual understanding in chemistry. This teaching and learning problem is embedded in the discipline of chemistry (which began as a descriptive one) making it particularly difficult to address. We have found that by VisChem animations and storyboards before shorthand symbols, students have a greater likelihood of building molecular-level models for phenomena.

Initial Findings: Findings are presented in the publications listed below.

Products:

  • Project Website
  • Publications
    • Magnone, K. Q., & Yezierski, E. (2024). Generating an evidence-based guide to scaffolding sodium chloride dissolution using the VisChem Approach. Journal of Chemical Education, 101(4), 1416–1424. https://doi.org/10.1021/acs.jchemed.3c00989
    • Magnone, K. Q., & Yezierski, E. (2024). Applying the VisChem Approach in high school classrooms: Chemical learning outcomes and limitations. Journal of Chemical Education, 101(3), 727-740. https://doi.org/10.1021/acs.jchemed.3c00827
    • Magnone, K. Q., & Yezierski, E. (2024). Beyond convenience: A case and method for purposive sampling in chemistry teacher professional development research. Journal of Chemical Education, 101(3), 718-726. https://pubs.acs.org/doi/10.1021/acs.jchemed.3c00217
    • Wu, M-Y M., & Yezierski, E. (2023). Investigating teacher-teacher feedback: Uncovering useful socio-pedagogical norms for reform-based chemistry instruction. Journal of Chemical Education, 100(11), 4224-4236. https://doi.org/10.1021/acs.jchemed.3c00409 
    • Wu, M-Y M., & Yezierski, E. (2023). Investigating the mangle of teaching oxidation-reduction with the VisChem approach: Problematising symbolic traditions that undermine chemistry concept development. Chemistry Education Research and Practice, 24, 807 - 827. doi: 10.1039/d2rp00321j
    • Wu, M-Y M., & Yezierski, E. (2023). Secondary chemistry teacher learning: precursors for and mechanisms of pedagogical conceptual change. Chemistry Education Research and Practice, 4, 245-262. https://doi.org/10.1039/D2RP00160H
    • Wu, M-Y M., & Yezierski, E. (2022). Exploring adaptations of the VisChem Approach: Advancements and anchors toward particle-level explanations. Journal of Chemical Education, 99(3), 1313–1325. https://doi.org/10.1021/acs.jchemed.1c01275
    • Wu, M-Y M., & Yezierski, E. (2022). Pedagogical chemistry sensemaking: a novel conceptual framework to facilitate pedagogical sensemaking in model-based lesson planning. Chemistry Education Research and Practice, 23, 287-299. https://doi.org/10.1039/D1RP00282A
    • Wu, M-Y M., Magnone, K., Tasker, R., and Yezierski, E. J. (2021). Remote chemistry teacher professional development delivery: Enduring lessons for programmatic redesign. Journal of Chemical Education, 98(8), 2518-2526. https://pubs.acs.org/doi/10.1021/acs.jchemed.1c00181
    • Accepted Paper: Mutch-Jones, K., & Storeygard, J. Building instructional capacity and creating opportunities for professional growth: Mathematics professional development for paraeducators. Mathematics Teacher Educator.

Suggested Reading: 

  • A must for anyone who wishes to have a framework for chemistry learning: Taber, K. S. (2013). Revisiting the chemistry triplet: drawing upon the nature of chemical knowledge and the psychology of learning to inform chemistry education. Chemistry Education Research and Practice, 14(2), 156-168.
  • A product of our project made possible by standing on the shoulders of Taber and the works cited in his 2013 paper: Wu, M-Y M., & Yezierski, E. (2022). Pedagogical chemistry sensemaking: a novel conceptual framework to facilitate pedagogical sensemaking in model-based lesson planning. Chemistry Education Research and Practice, 23, 287-299.
     


3DLP Logo

Developing and Testing a Learning Progression for Middle School Physical Science Incorporating Disciplinary Core Ideas

PI: Peng He 
Disciplines: Chemistry: Matter & its Interactions; Energy
Grade Levels: Middle School (Grades 6-8)

Project Description: This three-year project develops and tests three-dimensional learning progressions (3DLPs) with multiple pathways for middle school physical science. The 3DLPs incorporates the three dimensions of scientific knowledge from the Framework for K-12 Science Education: Disciplinary Core Ideas of “Matter and its Interaction” and “Energy”; the Science and Engineering Practices of “Constructing Explanations” and “Developing and Using Models”; and the Crosscutting Concepts of “Cause and Effect” and “Systems and System Models.” We employ a design-based research approach to iteratively design and test the 3DLPs for middle school physical sciences. The revision and testing process uses various data sources, including expert feedback, teacher and student interviews, classroom observations, and teacher and student artifacts. 

To address the need in teaching and learning physical science core ideas, we articulate the 3DLPs with multiple learning pathways that can support students from diverse backgrounds, especially those with low socioeconomic status and underrepresented populations, to develop a deeper understanding of middle school physical sciences. Participating teachers receive professional learning over time and guidelines on using the 3DLP to adapt their local curriculum and instruction materials. The project investigates how teachers use the 3DLP to improve their teaching and support and monitor students’ development of 3D understanding in physical science core ideas. 

Challenges in Physical Sciences Education & Strategies for Addressing Them: So far, we have identified two unique challenges related to teaching and learning physical science core ideas in middle school. Developing an integrated understanding of physical science core ideas with scientific practices (e.g., modeling) takes time. It requires teachers to monitor student learning along the 3DLP and provide feedback on students’ understanding over time. Analyzing students’ written and drawn responses in a timely manner would support teachers’ instruction to help students move to the next levels in the 3DLP. To address this challenge, this project developed rubrics with examples and provided professional learning to support teachers in using the rubrics. Another approach is to leverage generative AI tools to analyze students’ responses to provide meaningful information for teacher instructional decisions. This study is undertaking and will potentially gear up our Year 3’s teacher implementation. 

The second challenge focuses on teachers’ adaptation of their local curriculum materials using our 3DLP as a framework to guide their modifications. Teachers had challenges in modifying curriculum materials for two reasons: 1) They do not have sufficient expertise about the science standards (i.e., NGSS); 2) They tend to resist modifying their existing curriculum materials as much as possible as they know how to use them. In responding to this challenge, we provided mentorships for our participating teachers in two ways: co-designing and principle-based extended professional learning. In our co-design phase, we explicitly explained the importance of the 3DLP framework based on the developmental perspectives of learning and the notion of usable knowledge. Participating teachers and our research team work together to develop and revise the 3DLPs and adapt their local materials to 3DLPs. In addition, we tried to minimize teachers’ extra efforts by respecting their decisions about the degree of adaptation and offering constant and timely support during their implementation. Through co-designing and participating in the extended professional learning, teachers developed deeper understanding of d the developmental perspective of learning, the 3DLPs, and how to use 3DLPs in their classrooms. 

Innovative Technology Use: When teachers implemented the 3DLP materials, we found that teachers often quickly skimmed the whole class responses on assessment tasks, summarized the common challenges, and provided further support to improve students' learning about physical science. However, such strategies do not address students' learning needs, especially individual student's specific needs in a timely manner. Thus, we leveraged large language models (e.g., GPT models) to analyze students’ drawn responses to classroom assessment tasks and support teachers’ timely instructional decisions. Students' hands-on responses to our assessment tasks were analyzed based on analytic and holistic perspectives on three-dimensional learning performance. We conducted a trial experiment by creating a workflow using the GPT approach to analyze students' drawn responses to three assessment tasks. The preliminary results show that the GPT models could identify three dimensional knowledge across the patterns of analysis of the three tasks. The study design and the exploration of using generative AI as a partner will significantly contribute to the teaching and learning of physical science with innovative technologies.

Support for Equitable Teaching & Learning: This project supports students minoritized and marginalized racial and ethnic groups to learn physical science disciplinary ideas. We also recognize teachers’ limited efforts and resources to continually support students’ move forward to a more sophisticated understanding of physical science disciplinary core ideas. In our 3DLP, we co-developed our 3DLPs with urban, rural, and fringe school teachers. We articulated multiple learning pathways to provide diverse learning opportunities for our target students to advance from lower to upper levels in the learning progressions, supporting student learning of physical science core ideas. To support equitable teaching, we worked with teachers to adapt their local curriculum materials using the guidance of our 3DLPs to meet their students’ needs. In addition, we monitor focal groups of students' learning pathways across time to explore how teachers use the 3DLPs to modify their curriculum to advance student learning of physical science core ideas. 

Initial Findings: 

  • Student learning: We used mixed methods (Creswell & Plano Clark, 2018) to conduct quantitative and qualitative analysis to articulate students’ 3D understanding aligned with our 3DLP levels. We analyzed interview data from 30 students from three regions in Michigan with diverse backgrounds (e.g., ethnicity, gender, SES, and cultural and educational background). We found that students could achieve most individual levels of physical core ideas with scientific practices (constructing explanations) and crosscutting concepts (cause and effect) DCIs, SEPs, and CCCs based on our 3DLP. However, students had challenges in understanding the ideas of the temperature changes of different matter due to the nature of matter, which is an upper level in our 3DLP. 
  • Multiple Learning Pathways: We articulated multiple learning pathways in the 3DLPs to support diverse students’ understanding of physical science core ideas from lower to upper levels. For instance, when developing students’ 3D understanding of Matter and Its Interaction, some students may follow a learning pathway: properties of matter-atomic structures-chemical reactions-conservation of mass and atoms; whereas others may follow another learning pathway: properties of matter-chemical reactions-atomic structure-conservation of mass and atoms. 
  • Teacher learning: We collected and analyzed teachers’ pre-post implementation interview data. Our results showed that teachers developed their knowledge about 3D learning to explicitly support students’ modeling practices in understanding physical science core ideas. Moreover, by implementing 3DLP-based assessments in classroom, they developed their classroom practices of using assessments to promote students’ 3D learning. 

Products: Publications:

  • He, P., Zhang, Y., Li, T., Zheng, Y., & Yang, J. (2024). Diagnosing middle school students’ proficiency in constructing scientific explanations with the integration of chemical reactions and patterns: a cognitive diagnostic modeling approach. International Journal of Science Education, 1–28. https://doi.org/10.1080/09500693.2024.2413926
  • He, P., Shin, N., & Krajcik, J. (2024). Developing three-dimensional learning progressions of energy, interaction, and matter at middle school level: A design-based research. In Jin, H., Yan, D., & Krajcik, J. Handbook of Research on Science Learning Progressions. Routledge. doi: 10.4324/9781003170785-14
  • He, P. Shin, N. Kaldaras L., & Krajcik, J. (2024). Integrating artificial intelligence into learning progression-based learning systems to support student knowledge-in-use: Opportunities and challenges. In Jin, H., Yan, D., & Krajcik, J. Handbook of Research on Science Learning Progressions. Routledge. doi: 10.4324/9781003170785-31
  • He, P., Zhai, X., Shin, N., Krajcik, J. (2023). Applying Rasch measurement to assess knowledge-in-use in science education. In: Liu, X., Boone, W.J. (eds) Advances in Applications of Rasch Measurement in Science Education. Contemporary Trends and Issues in Science Education, vol 57. Springer, Cham. https://doi.org/10.1007/978-3-031-28776-3_13

Suggested Reading: 

  • He, P., Shin, N., & Krajcik, J. (2024). Developing three-dimensional learning progressions of energy, interaction, and matter at middle school level: A design-based research. In Jin, H., Yan, D., & Krajcik, J. Handbook of Research on Science Learning Progressions. Routledge. doi: 10.4324/9781003170785-14
  • He, P. Shin, N. Kaldaras L., & Krajcik, J. (2024). Integrating artificial intelligence into learning progression-based learning systems to support student knowledge-in-use: Opportunities and challenges. In Jin, H., Yan, D., & Krajcik, J. Handbook of Research on Science Learning Progressions. Routledge. doi: 10.4324/9781003170785-31

ChemLEAP Logo

Developing Learning Environments that Support Molecular-Level Sensemaking

PI: Ryan Stowe 
Disciplines: Chemistry
Grade Levels: High School

Project Description: We are thinking with teachers about how to make chemistry learning look more like daily life scenarios in which people and communities advance their goals by thinking about and with molecules. This involves re-thinking what and whose knowledge is centered in our courses and broader ways of thinking about what “success” might look like. Our field commonly argues that science classes should prepare learners to reason about and with science in their post-school daily lives. In order to advance this goal, instructional resources and support for professional development must move beyond focusing on canonical correctness and instead make space for learners to take up ways of knowing and learning (i.e., epistemologies) that are potentially useful in-life. This project explores how a community of teachers and researchers can collaboratively build chemistry instructional resources that signal the utility of epistemologies one might make use of in daily life. This work requires theory building (e.g., what features of a learning environment lead students to compile their epistemologies in a given way?), design of professional learning opportunities, and curriculum co-development. Products of this work include scholarship on teachers’ pedagogical sensemaking about curriculum-embedded epistemological messages, ways of eliciting students’ experiences with these messages, and curricular materials for a 2-semester high school chemistry course. The ChemLEAP teacher learning community meets regularly to discuss tools and practices productive for making chemistry learning more useful. Recently, we have begun attending to ways in which chemistry knowledge and ways of knowing might empower communities to advocate for more just social conditions.

Practices Supported: We have worked with teachers to support assessment of their epistemic learning goals for students. These are goals about how students view knowing and learning rather than their knowledge of some pre-specified science fact or skill. An example of one such goal is "Students will be able to make sense of a natural phenomenon by engaging in whichever science practices or other reasoning strategies are useful to them in the given context." In this example, students view learning as a sensemaking process, guided by their own (or their learning community's) critical judgement about what thinking tools to employ and what counts a "good" or "useful" knowledge. Our teacher colleagues work to assess this goal by presenting students with complex, novel scenarios and providing them with scaffolded opportunities to construct arguments and pose questions in order to better understand the scenario.

Challenges in Physical Sciences Education & Strategies for Addressing Them: When you teach high school physical science, you are pulled in a lot of different directions. You may have to comply with departmental learning objectives, standardized metrics for “success”, expectations of college STEM coursework, etc. We think a lot about how we can have substantive impacts on what knowing and learning means in our classes in light of these many constraints. Through our work, it has become clear that high-stakes assessments must be part of the conversation if we are to open space for a variety of epistemologies in chemistry classes. If we have expansive, agentic, in-class conversations but tests are all about recalling science facts, student will experience our class as mostly about recalling canon.

Support for Equitable Teaching & Learning: Through conversations among members of our community as well as dialogue with friends and colleagues in Science Education and Science & Technology Studies, it has become clear that storylines curricula, such as ChemLEAP, do not go far enough in centering the priorities and perspectives of students and their communities. While such curricula may occasionally open space for students to surface and build on their lived experiences, the primary aim of storylines courses remains engaging with science knowledge and practices in ways that appear normative. We recognize that centering justice and equity requires a more expansive view that includes ways power and oppression shape how chemistry knowledge is manufactured and used. Doing so will let us regularly grapple with questions like “who benefits?” and “who is harmed?” when we think about extraction, purification, commodification, and disposal of chemical substances. Answers to these questions, we claim, are central to a vision of school chemistry that is useful in daily life. 

Initial Findings: We have constructed an analytic model for how students’ engagement with a semester-long course affects ways of knowing and learning they experience as useful. This modeling work is also supporting development of an approach to design-based research that centers epistemological responsiveness (vs. canonical correctness). In addition, we have theorized about how curriculum-embedded epistemological messages may constrain how and why teachers enact curricular materials in certain ways.
 
Ongoing work is exploring how different ChemLEAP enactments communicate to students what and whose knowledge counts in and across moments. These investigations are letting us explore how (or whether) course-embedded epistemological messages relate to the equity and justice goals we aspire towards. We are also thinking with our instructor colleagues about how their intended messages about useful knowledge and ways of knowing map onto students’ experiences with these messages (or not!). Doing so lets members of our community refine their materials and practice toward desired epistemic learning goals. 

Finally, we hope to investigate in a future project how professional learning communities can support teachers to center justice in their classrooms. We are eager to leverage the knowledge and relationships that we've built with our collaborating teachers to keep moving towards a vision of chemistry education that is not only empowering to students and their communities, but also potentially emancipatory. 

Products: 

  • ChemLEAP Curricular Materials
  • Schafer, A.G.L., Kuborn, T.M., Schwarz, C.E., Deshaye, M.Y., & Stowe, R.L. (2023). Messages about valued knowledge products and processes embedded within a suite of transformed high school chemistry curricular materials. Chemistry Education Research and Practice, 24, 71-88.
  • Schwarz, C.E., DeGlopper, K.S., Greco, N.C., Russ, R.S., & Stowe, R.W. (in press). Modeling student negotiation of assessment-related epistemological messages in a college science course. Science Education.

Suggested Reading:

  • Morales-Doyle, D. (2024). Transformative science teaching: A catalyst for justice and sustainability. Harvard Education Press.
  • Tolbert, S., Wallace, M. F., Higgins, M., & Bazzul, J. (2024). Reimagining Science Education in the Anthropocene, Volume 2 (p. 421). Springer Nature.
  • Tan, E., & Barton, A. C. (2023). Teaching toward rightful presence in middle school STEM. Harvard Education Press.

Auto Feedback 3DLP Logo

Evaluating Effects of Automatic Feedback Aligned to a Learning Progression to Promote Knowledge-In-Use

PI: Kevin HaudekCo-PIs: Leonora Kaldaras, Joseph Krajcik
Disciplines: Physical Sciences: Structure and Properties of Matter; Forces and Types of Interactions; Energy
Grade Levels: 9-12

Project Description: This project develops and explores the impact of an artificial intelligence (AI) assessment system that automatically generates feedback based on students' open-ended responses in physical science, aligned with established learning progressions on electrical interactions. Learning progressions are cognitive models that are based on a developmental approach to learning. The AI scoring system will offer cognitive, tailored feedback to students based on their responses and provide teachers with class summaries. Such tools support formative assessment within high school physical science that meets the Next Generation Science Standards' performance expectations, aimed at enhancing student learning.

The AI system uses natural language processing, image recognition, and supervised machine learning to evaluate students' written explanations and digitally created models. The feedback is tailored to the learning progression and delivered according to the AI's classification of the responses. The study assesses whether this automated feedback enhances students' learning outcomes and their progression regarding electrical interactions. Additionally, the project fosters scientific knowledge by involving students in scientific practices like modeling, connecting core disciplinary concepts, and using crosscutting ideas to analyze compelling phenomena, with real-time feedback provided throughout the process.

Practices Supported: We focus on the scientific practices of Constructing Explanations and Developing and Using Models and assess student learning at both macroscopic and microscopic levels of understanding. We have students Develop and Use Models as this practice is central to developing deep understanding in science, and cannot be assessed by explanation type items alone. To demonstrate proficiency in scientific modeling, students need to represent relevant components, relationships between components and connections to a given phenomenon. We ask students to Construct Explanations in response to simulations and demonstrations within the curriculum. The purpose of the explanation is to support reasoning about the observed phenomena using the relevant DCI of forces and electrical interactions. We support student explanation building by using a claim-evidence-reasoning framework in the curriculum.

Challenges in Physical Sciences Education & Strategies for Addressing Them: One of the important challenges we have faced in the current project is simultaneous analysis of student performance on multi-modal assessments aligned to a knowledge-in-use LP. An example of an LP-aligned multi-modal assessment item is asking students to develop a scientific model explaining an electrostatic phenomenon and write a short accompanying explanation reflecting their reasoning. Such assessments are challenging to evaluate using AI due to the need to simultaneously evaluate the model, the explanation, and make a decision on final LP-level placement. Moreover, the resulting feedback should provide productive and constructive comments to specific ideas present in the student response in order to support the student developing more sophisticated understanding as described by the LP. In the current project, we have evaluated the two modalities separately using AI, and then humans make decisions regarding the final LP level placement and feedback based on the two modalities. In the future, we hope to explore training AI to evaluate student performance on both modalities simultaneously and provide the final LP level placement and feedback based on information from both modalities. We believe this will allow us to streamline the process of evaluation and feedback on multi-modal LP-aligned assessments measuring knowledge-in-use.

Innovative Technology Use: Natural language processing, generative artificial intelligence, supervised machine learning

Support for Equitable Teaching & Learning: We pay special attention to equity in this project in several ways. First, the student response data we use to train the AI algorithms in this project is collected from a diverse range of high schools, including rural, urban and high-need school districts with a high percentage of learners from backgrounds historically marginalized in STEM and as multilingual learners (MLLs). Therefore, the language in student responses used to train AI is diverse, often non-standard, and reflects a wide range of ways of knowing and engaging with the content. We train our human scorers to recognize this diversity and evaluate it with an equity-centered lens. For example, when training human coders to look for evidence of students using ideas from Coulomb’s law in their responses, we help coders understand and identify the different non-standard and non-academic ways of explaining ideas using Coulomb’s law. That diversity is also captured in our scoring rubrics, and by our trained scorers, and it is ultimately recognized by AI because we explicitly train the AI to recognize this diversity. Second, our rubrics allow us to provide tailored, LP-aligned constructive and productive feedback to individual learners on their diverse ways of thinking because we capture this diversity through specific analytic rubric categories that guide the development of feedback provided to individual learners. Third, we employ an iterative approach on constantly evaluating AI-based scores and AI-human misscores to specifically focus on the possible reasons for miscores and relate these misscores to possible issues on equity and bias in AI algorithms. 

Cross-Curricular Connections: The Interactions curriculum engages students in exploration of driving questions, experiencing and making sense of multiple phenomena, fosters student collaboration, and results in student development of artifacts to represent their learning. Unit 1 in the curriculum supports students building macroscopic level understanding of electrical interactions by having students explore why some clothes stick together when they come out of the dryer. Students investigate patterns in how charged objects interact, build macro-level understanding of Coulomb relationships, and use these ideas to construct explanations of electrostatic phenomena. Unit 2 expands the idea of energy and supports students in building an understanding of how energy changes in a system affect electrical interactions between objects in the system at both macro and atomic/molecular levels. Students explore how a small spark can start a huge explosion. First, students figure out how energy transfers between potential and kinetic, and ultimately develop an explanation for how energy is involved in interactions between atoms as it relates to bond formation. 

Initial Findings: We have designed and tested a learning progression guided approach to training AI algorithms to evaluate complex science understanding reflective of knowledge-in-use in different modalities: text-based explanations and drawing-based scientific models. Preliminary results indicate that this approach helps develop rubrics for human and AI-based scoring that yield high human-human and human-AI agreement on both modalities. These results suggest promise in training AI algorithms to evaluate complex reasoning and skills in a way that is grounded in relevant, empirically validated learning theories, reflecting human-in-the loop approach to AI training.

Products: 

  • Project Website
  • Presentations & Publications
    • Kaldaras, L., Li, T., Haudek, K., & Krajcik, J. (2024, April 12). Developing rubrics for AI scoring of NGSS (Next Generation Science Standards) learning progression–based scientific models. Paper presented at the 2024 Annual Meeting of the American Educational Research Association. doi: 10.3102/2109181
    • Li, T., Kaldaras, L., Haudek, K., & Krajcik, J. (2024, April 11). Using AI to evaluate multimodal formative assessments in physical science. Poster presented at the 2024 Annual Meeting of the American Educational Research Association. doi: 10.3102/IP.24.2112964
    • Kaldaras, L., Haudek, K., & Krajcik, J. (accepted for publication). “Challenges and Promises of Employing Automatic Analysis Tools Aligned To Learning Progressions To Assess Knowledge Application and Support Learning In STEM”. International Journal of STEM Education.


NoriLLA LogoIntelligent Science Stations: Developing Adaptive Mixed-Reality Technology to Enhance Inquiry-based STEM Learning in Schools

PI: Nesra Yannier | Co-PIs: Scott Hudson, Ken Koedinger
Disciplines: Physical Science, Stability and Balance, Critical Thinking, Forces and Motion
Grade Levels: PreK-5

Project Description: We have been developing and researching Intelligent Science Stations, a new genre of interactive science experiences that add an AI layer on top of physical experimentation to improve children’s STEM learning and engagement. The motivation behind this project is the need for engaging, inquiry-based science learning opportunities for young children in the classroom, to sustain early interest in science. The Intelligent Science Stations provides students with hands-on science experiences, augmented by an intelligent agent that offers feedback based on artificial intelligence (AI) computer vision. These Intelligent Science Stations incorporate scientific apparatuses such as earthquake tables, ramps or balance scales, and students' actions are observed and evaluated by the intelligent agent. The agent appears as an animated character on a screen and provides interactive feedback and guidance to guide students through scientific inquiry. This innovative approach offers evidence-based, personalized support and feedback to children, while also assisting teachers in integrating more inquiry-based science learning into their classrooms. By modeling behaviors like asking questions, making predictions, and explaining scientific phenomena, the interactive AI system helps teachers enhance their classroom experiences.

Students engaged with NoriLLA

Practices Supported: Intelligent Science Stations can be used to teach different STEM concepts such as early physics, balance and stability, forces and motion, scientific inquiry, as well as 21st century skills like critical thinking, collaboration and persistence.

Innovation in Assessing Teaching Practice: There are currently no instruments that measure teacher practice of paraeducators. We expect that the instruments we have developed will be useful to the field.

Challenges in Physical Sciences Education & Strategies to AddressThem: Based on our research, we have seen that many teachers especially in elementary school, don’t have a science background so are afraid to teach science concepts. They want technologies that support them together with the children. Here is what one of the teachers said after seeing our system: “I love that NoRILLA uses technology in such an engaging, communicative and non-isolating way. I'm not a scientist, I'm not a scientist by any stretch of imagination and I love science and I love to teach science, but I feel like I'm limited by own limitations in the science world. To have something like this that supports and backs up and lets the kids and myself all learn together is genius!"

Innovative Technology Use: We have a patented AI and mixed-reality system that uses AI computer vision to track what children are doing in the real world as they do scientific experiments and provides personalized, adaptive, interactive feedback.

Support for Equitable Teaching & Learning: We work with many community organizations that reach underserved students such as Boys and Girls Clubs, Head Start programs and school districts that serve children from low-income backgrounds. We make sure our curriculum and professional development materials can support a wide-range of backgrounds.

Cross-Curricular Connections: We have been developing an extensive STEM curriculum around our system in collaboration with teachers. Based on the feedback from the teachers, we have made our curriculum and lesson plans cross curricular and interdisciplinary so they can be used to teach many different concepts including Early Physics, Stability and Balance, Inquiry, Symmetry, ELA (English Language Arts), Ratio, Maker, Engineering, Math, Graphing, Geometry, Literacy and Geography.

Initial Findings: Early research has shown that AI-enhanced intelligent science stations improve children’s STEM learning by 5 times compared to other screen-based technologies. Also, adding an AI-based guided inquiry layer on top of physical experimentation improves children’s learning and engagement dramatically compared to traditional maker spaces and hands-on exhibits (4x dwell time). 

Our recent research in schools has shown that children’s informed decision making and deep understanding of standards-aligned STEM concepts doubled in one year using NoRILLA Intelligent Science Stations. We also saw that there was a 32% increase in students’ ability to Explain their answers, which relates to Higher Level Thinking as highlighted in NGSS and common core standards. Looking at grades separately, we saw that a kindergartner interacting with NoRILLA for 3 months significantly exceeded the scientific understanding of a second grader (3 times the natural yearly progression). 
 

Products:

  • Project Website
  • Yannier, N., Hudson, S. E., Koedinger, K. R., Hirsh-Pasek, K., Golinkoff, R. M., Munakata, Y., ... & Brownell, S. E. (2021). Active learning:“ Hands-on” meets “minds-on”. Science, 374(6563), 26-30.

Suggested Reading: 

  • Yannier, N., Hudson, S. E., & Koedinger, K. R. (2020). Active learning is about more than hands-on: A mixed-reality AI system to support STEM education. International Journal of Artificial Intelligence in Education, 30, 74-96.
  • Yannier, N., Crowley, K., Do, Y., Hudson, S. E., & Koedinger, K. R. (2022). Intelligent science exhibits: Transforming hands-on exhibits into mixed-reality learning experiences. Journal of the Learning Sciences, 31(3), 335-368.


STEP UP LogoMobilizing Physics Teachers to Promote Inclusive and Communal Classroom Cultures Through Everyday Actions

PIs: Zahra Hazari, Beth Cunningham, Claudia FracchiollaCo-PIs: Justine Harren, Geoffrey Potvin, Annelise Roti Roti, Pooneh Sabouri, Nicole Schrode
STEM Disciplines: Mathematics
Grade Levels: Secondary

Project Description: STEP UP (Supporting Teachers to Encourage the Pursuit of Undergraduate Physics) Physics (stepupphysics.org) is a national community of physics teachers, researchers, and professional societies that have come together to help address a persistent and challenging problem in physics: the historic and continued marginalization of women and minoritized racial/ethnic groups. STEP UP teachers implement two research-based lessons (which present counternarratives about who does physics and what counts as physics) and practices from the Everyday Actions Guide (EAG) (a resource that supports teachers in taking actions that foster classroom cultures that are inclusive, responsive, and communal). These resources have demonstrated having a significant positive effect on the physics identity and persistence of women and minoritized racial/ethnic groups. While sufficient support exists for teachers to implement the lessons, there is a critical gap in the support needed for teachers to implement the EAG such that they transform the culture of their classrooms. This project focuses on designing, testing, and broadly implementing Professional Learning (PL) that will support both pre-service and in-service physics teachers to adopt everyday actions that foster inclusive and communal classroom cultures. Through the project’s activities, evidence-based professional learning and community support resources will be provided to hundreds of high school physics teachers that will, in turn, impact the physics learning opportunities and environment for thousands of students. These efforts will help to engage and inspire students, particularly women and minoritized racial/ethnic groups, by shifting the culture of physics from an individualistic, competitive, and abstract enterprise to one that is communal, interdependent, and connected.

Practices Supported: The project is designing Professional Learning (PL) opportunities for high school physics teachers in order to help support their adoption of everyday actions that foster inclusive and communal classroom cultures. The PL design draws on best practices for teacher learning and is co-designed by high school physics teachers, science teacher educators, and science education researchers. Furthermore, the PL includes components for both pre-service and in-service teachers (PL Summit), as well as components specifically for in-service teachers throughout the school year (PL Teams).

Challenges in Physical Sciences Education & Strategies for Addressing Them: The project was initiated as a response to challenges faced by high school physics teachers who required more adaptive PL and community support to enact everyday actions that would help transform their classroom culture to be more inclusive/communal. Connecting and supporting isolated high school physics teachers (on average, only one per school) remains a challenge. However, the STEP UP community and team of physics teachers, researchers, and professional societies is the most well situated to take on this challenge and respond to the needs of physics teachers and their students, especially those who continue to be marginalized. 

In terms of research, a challenge will be to carefully test the impact of the PL on physics teachers and their students. This will be done through a study with pre-service and in-service teachers randomly assigned to treatment and control groups, the former who experience the PL and the latter who do not. The impact on teachers’ beliefs and practices will be compared across groups, as well as the impact on students’ perceptions of classroom culture, physics identities, and intentions to persist in physics. Implementing this type of study is always challenging but ensures that the PL is having the impact it was designed for!

Support for Equitable Teaching & Learning: The project focuses on fostering a communal culture within the physics classroom.  This culture includes both social learning that is interdependent and critical caring that is responsive and inclusive. Furthermore, it resists the dominant narrative of physics as individualistic (for geniuses, not focused on benefiting others/society), competitive, and abstract (lacking meaningful connections for students). Our research findings show that experiencing communal cultures in high school physics has a significant effect on the physics identities and persistence of students, particularly those marginalized in physics. Stay tuned for these articles!

Cross-Curricular Connections: The Everyday Actions Guide (EAG) consists of five sections (interacting with students individually, facilitating group work/labs, engaging the whole class, planning and assessing, expanding outside the classroom), and within each section, multiple pedagogical/curricular strategies that foster personal/cultural/community/interdisciplinary connections with physics, as well as the creation of a communal classroom culture.

Products:

  • Project Website
  • Project Video
  • Publications: 
  • Presentations:
    • Barnett Dreyfuss, B. (2024). Working with counselors to strengthen your physics programs. American Association of Physics Teachers (AAPT) Summer Meeting, July 10, Boston, MA.
    • Barnett Dreyfuss, B., & Johnson, J. (2024). Inclusive actions for physics classrooms of all levels. American Association of Physics Teachers (AAPT) Winter Meeting, January 9, New Orleans, LA.
    • Barnett Dreyfuss, B., Bowns-Kamphuis, K., Garland, C., & Poulan, P. (2024). An Introduction to STEP UP. American Association of Physics Teachers (AAPT) Summer Meeting, July 8, Boston, MA.
    • Bowns-Kamphuis, K. (2024). Strengthening your physics programs: Making counselors & families alies. American Association of Physics Teachers (AAPT) Summer Meeting, July 10, Boston, MA.
    • Bowns-Kamphuis, K. (2024). STEP UP Careers in physics & growing your physics program. National Science Teaching Association (NSTA) National Conference on Science Education, March 23, Denver, CO.
    • Hazari, Z. (2024). Towards a Communal and Responsive Culture in High School Physics. Symposium on “Working on Equity in Science Education Across Places and Spaces”, National Association for Research on Science Teaching (NARST) Annual Conference, March 19, Denver, CO.
    • Hazari, Z. (2024). STEP UP: An NSF DRK-12 journey. American Institute of Research EQR Hub: Community of Practice Series, April 2, Virtual.
    • Mateu, C., Sabouri, P., Hazari, Z., & Fracchiolla, C. (2024). Examining communal classroom cultures: Comparing physics teachers’ and students’ perceptions. American Association of Physics Teachers (AAPT) Summer Meeting, July 8, Boston, MA.
    • Mateu, C., Sabouri, P., & Hazari, Z.(2024). Examining communal classroom cultures: Comparing physics teachers’ and students’ perceptions. Physics Education Research Conference (PERC), July 10, Boston, MA.

Additional Projects

We invite you to explore a sample of the other recently awarded and active work with a focus on chemistry and/or physics education.


Related Resources

Year