Analyzing & Interpreting Data Across STEM Disciplines

The demand for data skills is growing across industries and disciplines. Understanding how to analyze and interpret data will not only be critical for students as they enter the workforce, but also as they grapple with complex issues, such as the COVID-19 pandemic or climate change. This Spotlight features 16 DRK-12 projects that are building students' data skills through research and innovative curriculum and tools.

In this Spotlight...


Featured Projects

Building Students' Data Literacy through the Co-design of Curriculum by Mathematics and Art Teachers (Collaborative Research)

(PIs: Camillia Matuk, Megan Silander, Ralph Vacca; Co-PI: Kayla DesPortes)
Grades: 6-8
Target Audience: Middle school students and middle school art, dance, and mathematics teachers.

Disciplines/Subject Areas: Disciplines covered include: mathematics, art, dance, and data science. Specific practices include: interpreting and reasoning about data, communicating about data, making evidence-based claims, creating persuasive arguments through visual representation and dance.

Project Description: This project aims to explore art-integrated approaches to broadening middle school students’ data literacy and participation in data science. By partnering with math and arts teachers, we are co-designing and testing classroom-based curricula that build on the practices that are common and unique between art and data science, to help students reason about social issues that are relevant to their lives, and to create new ways for them to engage in data science. Outcomes from this project will include a refined understanding of the interdisciplinary co-design process, and of the learning opportunities in arts-integrated data science activities.

Describe the data that students work with in your project: The data students will work with are related to social and environmental issues, and will include a mix of student-generated and professionally-collected data. In our most recent implementation, for example, students documented their social interactions over the course of a day, and also explored existing data sets on teenagers’ use of social media. Because researchers are codesigning curricular units in collaboration with teachers, the data students work with will depend on teachers’ and students’ interests. Topics we are considering in designing our upcoming  modules include data related to students’ personal study habits, neighborhood quality, and the health and economic impacts of COVID-19.

We will be drawing on publicly available data from sources such as the Pew Research Center, NYC OpenData, Spotify,data.gov, and university data repositories such as those maintained by the University of California, Irvine and the University of Michigan.

Initial Findings Related to Analyzing & Interpreting Data: We conducted a pilot study of a 3-week long unit, implemented in Spring 2020 with one classroom led by a math and art teacher pair. Our goals for this pilot study were to explore how students use mathematics and art to make data-based claims, and to examine the skills in the arts, mathematics and data literacy evident in their work.

The unit we co-designed with teachers was conducted online and asynchronously because our partner school closed during the pandemic. The unit guided students in two activities that culminated in data-driven artworks. The first activity had students generate, collect, and analyze data to answer questions about their lock-down behaviors (e.g., with whom they had had conversations, or how many times and where they heard the words “COVID,” “pandemic,” “coronavirus,” and other related words over the course of a day) and represent these in a drawing. In a second activity, we asked students to explore a dataset from the Pew Research Center about teens’ uses of social media, identify key relationships, and then create data sculptures from everyday materials in their homes to communicate a story about those data...Read more.

Instruments: We piloted pre- and posttest assessments of students’ engagement with data science by adapting items from instruments developed by the Learning Activation Lab (activationlab.org/tools). We have also developed and piloted new assessment items to capture how students use art as a disciplinary lens in learning data science concepts and practices. These items assess students’ abilities to critique professionally created data-driven artworks in terms of their artistic choices and the message communicated about the data, and to raise questions about issues with data including sampling, quality, and bias.

Key Challenge: A major challenge in this work is developing interdisciplinary approaches to data science that are authentic to each of mathematics, art, and data science. A second challenge is in moving from typical data analysis activities that focus on measures of central tendency, distribution and frequencies, toward informal data inference skills. A third challenge is in developing ways to support our teacher partners who may be uncomfortable teaching with data on politicized issues that they identify as engaging for their students (e.g., mask-wearing, policing). A fourth challenge is adapting this project for a remote format, such that there are comparable opportunities for collaborative learning through discussion, for equitable access to technology and art materials, and for presenting work to an authentic audience.

Product(s): Products from this project will include classroom-ready arts-integrated curriculum modules; documents of our co-design process to inform educators and researchers looking to embark on similar interdisciplinary co-design projects; and publications of our research findings on the opportunities for student data literacy learning through the arts, and for teachers' professional learning through interdisciplinary co-design.


CAREER: Engaging Elementary Students in Data Analysis through Study of Physical Activities

(PI: Victor Lee)
Grade Level: 5-6
Target Audience: Students in a rural community

Disciplines/Subject Areas: Statistics

Project Description: There is a pressing need to prepare students to work with data, but often the data that they encounter in school are far removed from their daily experience. This project tries to build on their daily experience by using new technologies – for instance, equipping wearable devices, primarily activity trackers and heart rate monitors or high speed cameras – to collect data throughout the school day. We design and test curriculum and tools to help students access their data and analyze it in order to develop their data analysis and statistical reasoning skills. Our focus is upper elementary students, but we have done some explorations with high school students as well to test some configurations and tools out as well. In service of this design work, we also look at data practices in other settings to understand what routines and activities could be productively integrated into elementary curricular experiences.

Describe the data that students work with in your project: Students use wearable devices to obtain data about their own school day.

Initial Findings Related to Analyzing & Interpreting Data: Data from students’ personal experiences are highly engaging and position students in a more expert role than their classroom teachers. This changes a lot of the classroom dynamics. It is possible to make significantly greater progress on statistical reasoning using students’ own data compared to teaching about data using traditional approaches. Students appear to comprehend activity data representations quite readily with little explicit instruction through students sharing and discussing their own data. This seems to result from being highly familiar with the activities that generated those data.

Key Challenge: Commercial technologies that obtain activity data (e.g., Fitbit devices) can be challenging to use for classroom purposes because they limit 3rd party data access, although those technologies are familiar and motivating to students. Device form factor is an important consideration for youth to use data collection devices. Substantial support is required for students and classrooms to design feasible experiments involving activity data. Our approach has been to emphasize use of already collected activity data and to emphasize comparison of conditions.

Products: Video

Publications:

Lee, V. R. (2019). On researching activity tracking to support learning: a retrospective. Information and Learning Sciences, 120(1/2), 133-154. doi:10.1108/ILS-06-2018-0048

Lee, V. R., Drake, J. R., & Thayne, J. L. (2016). Appropriating quantified self technologies to support elementary statistical teaching and learning. IEEE Transactions on Learning Technologies, 9(4), 354-365. doi:10.1109/TLT.2016.2597142

Lee, V. R.(2015). Combining high-speed cameras and stop-motion animation software to support students' modeling of human body movement. Journal of Science Education and Technology, 24(2-3), 178-191. doi: 10.1007/s10956-014-9521-9

Lee, V. R., Drake, J., & Williamson, K. (2015). Let’s get physical: K-12 Students using wearable devices to obtain and learn about data from physical activities. TechTrends, 59(4), 46-53. doi: 10.1007/s11528-015-0870-x

Lee, V. R.(2013). The Quantified Self (QS) movement and some emerging opportunities for the educational technology field. Educational Technology, 53(6), 39-42.

Lee, V. R., & Drake, J. (2013). Quantified recess: Design of an activity for elementary students involving analyses of their own movement data. In J. P. Hourcade, E. A. Miller & A. Egeland (Eds.), Proceedings of the 12th International Conference on Interaction Design and Children 2013 (pp. 273-276). New York, NY: ACM.

Lee, V. R., & Drake, J. (2013). Digital physical activity data collection and use by endurance runners and distance cyclists. Technology, Knowledge and Learning, 18(1-2), 39-63. doi: 10.1007/s10758-013-9203-3

Lee, V. R., & Thomas, J. M. (2011). Integrating physical activity data technologies into elementary school classrooms. Educational Technology Research and Development, 59(6), 865-884. doi: 10.1007/s11423-011-9210-9


CAREER: Supporting Model Based Inference as an Integrated Effort Between Mathematics and Science

(PI: Ryan Jones)
Grade Levels: 6-7
Target Audience: The partner school is located in a small, southeastern town in the United States and serves a population of families from diverse ethnic, language, and economic backgrounds, including immigrant families from Central American and Southeast Pacific countries.

Disciplines/Subject Areas: We aim to develop an integrated approach to data in middle grades science and math classrooms that supports students to develop competencies in data visualization, statistics, and modeling and to use these to make inferences about ecological phenomena.

Project Description: This project is exploring how to productively coordinate instruction around data, statistics, modeling, and inference in middle grades mathematics and science classes. We will conduct design-based research to develop and study innovative tools that support students to generate knowledge about ecological systems by using models of variability to make inferences. In partnership with teachers from our partner school, we will develop two investigations in 6th and 7th grade science classes that will be coordinated with instruction in mathematics classes around data. Our research will focus on developing design principles for coordinating instruction, evidence about how students use ideas from mathematics classes to make inferences in science class, and evidence about how new questions in science class provoke a need for new mathematical tools.

Describe the data that students work with in your project: Students in our project will create the data used for instruction and their inquiry. However, the nature and source of the data will differ based on the type of variation that the unit focuses on. Students will collect data to explore variation from measurement error, variation across time, natural variation across units, and variation between different conditions. They will create these data from both wild space on their school property, Wisconsin Fast Plants they grow in their classrooms, and probability models they develop to inform their inferences.

Initial findings: This project recently began in the spring of 2020. We anticipate that our project will contribute knowledge about how to help teachers support interdisciplinary learning goals as a collaborative effort across math and science, how students make use of mathematical ideas as epistemic tools to generate knowledge, and how new questions about ecological systems motivate a need for new mathematical tools. We are finding early on that key features related to developing students’ data competencies (such as type of variation and data, dimensionality, and mathematical ideas embedded in data models) may be invisible to science teachers that orient their instruction around questions for inquiry.

Instruments: We will be making use of assessment items developed by Richard Lehrer and Mark Wilson. This item bank has been calibrated to 7 constructs related to data and statistics. Teachers will use the items for formative assessment and we will use student responses for research.

Key Challenges: Coordinating instruction related to data across math and science is challenging because schools are built to separate disciplines more than coordinate them, and science and mathematics communities use the similar words in their goals, but often have different understanding or vision for data instruction. So, developing interdisciplinary practices, like making inferences with data, across multiple years of schooling is the biggest challenge we face as a project.

Products: This project’s funding recently began in the spring of 2020. We anticipate developing three types of products: 1) A design framework for coordinating disciplinary learning goals in math and science around the interdisciplinary practice of making inferences with data, 2) four integrated investigations for 6th and 7th grade, and 3) Exemplars of students’ reasoning as they create, revise, and use models of variability to make inferences about ecological systems.


Case Studies of a Suite of Next Generation Science Instructional, Assessment and Professional Development Materials in Diverse Middle School Settings

(PI: Nancy Butler Songer)
Grades: 6-8
Target Audience: Middle school students (6th, 7th and 8th grades) utilizing remote learning and affiliated with under-resourced urban schools in Philadelphia, PA and Los Angeles, CA.

Disciplines/Subject Areas: Science (see NGSS Alignment)

Project Description: The global pandemic and climate change have led to unprecedented environmental, social, and economic challenges with interdisciplinary STEM foundations. Even as STEM learning has never been more critical, few instructional programs prepare students to apply classroom learning to the engineering design of solutions. This project focuses on designing and evaluating a phenomena-centric curricular unit where middle school student learning is used in the generation of solutions to increase indigenous populations of insects within local urban neighborhoods. The learning approach, open-source STEM, emphasizes learning content through the science and engineering practices and learning contributing to solutions to address local socioscientific issues. The materials are designed for middle school students and teachers in culturally, racially, and linguistically diverse, under-resourced schools in Philadelphia and Los Angeles.  Case study research provides results on students’ and teachers’ learning and changes in classroom practice.

Describe the data that students work with in your project: There are five related kinds of data on local insect populations that students collect, analyze, interpret, and use as evidence for solutions and arguments. These data support a more general understanding associated with NGSS MS-LS2-1 Analyze and interpret data to provide evidence for the effects of resource availability on organisms and populations of organisms in an ecosystem.

  1. Data on local animals to use as evidence within an argument to address the scientific question, is this local animal an insect?
  2. Data on local animals to use as evidence within an argument to address the scientific question, what does my insect eat and what eats my insect?
  3. Data on local animals to use as evidence within an argument to address the scientific question, what happens to my insect if the habitat (biology or physical) is disrupted?
  4. Data on local animals to use as evidence within an argument to address the scientific question, why are insects important?
  5. Data to support or refute the feasibility of solutions designed to increase populations of local indigenous insects in their neighborhood.

Key Challenges: We call our challenge Strategic Simplification. Our project is guiding middle school students in 3D learning about biodiversity. We are also guiding them to build on their data analysis and interpretation to design solutions to address local, ill-structured socioscientific issues. Strategic Simplification is the lengthy, transdisciplinary process of dialogue, trial-and-error, and productive failure that involves every member of our team: scientists, educators, software designers, and researchers. Strategic Simplification is the work to create age-appropriate, problem-solving, and engineering design activities that respect the science and integrity of the ill-structured problem while also engaging and appropriately challenging for student learning.

Products: We have just completed year one and we will implement our first cycle of research beginning October 1, 2020. We are happy to share curriculum modules and early results after that research cycle.


Climate Change Narrative Game Education (CHANGE)

(PI: Glenn Smith)
Grades: 9-12
Target audience: High school students, and their teachers, particularly for marine sciences courses, but high school students in any natural sciences courses.

Disciplines/Subject Areas: Marine Sciences, natural sciences, biology, physics, biology and engineering.

Project Description: This project helps high school students learn complex Global Climate Change (GCC) science by making it personally relevant and understandable. CHANGE provided a curriculum, integrated it into elective Marine Sciences high school courses, and tested its efficacy. CHANGE uses: (a) scientifically realistic text narratives  (a web-based science fiction novel with computer games) about future Florida residents (text stories with local Florida characters, 50-100 years in the future based on GCC), (b) local, place-based approach grounded in west-central Florida Gulf Coast and (c) simulations & games based on scientific data to help students learn principles of GCC so students can experience and cope with the potential long term effect of GCC via role-play and science-based simulation, and (e) a web-based intermedia eBook narrative where sections of narrative text alternate with simulations/computer games. The project has been used in 27 high schools, in Marine Science courses, in Hillsborough County, Florida.

Describe the data that students work with in your project: The data comes from a variety of sources, including from a geologist and from a variety of government agencies such as NOAA. Students also collect and work with their own data. For example, in one of the units the students engage in authentic science research on the effects of climate change on the growth of algae and relate that to red tide. And one of the games uses data from the research of one of our co-PIs (Ping Wang), who does research on coastal sediments.

Initial Findings Related to Analyzing & Interpreting Data: In this research, we explored how a prototype curriculum, Climate Change Narrative Game Education (CHANGE), helped students learn and gain interest in complex Global Climate Change (GCC) science by making it personally relevant and understandable. This research was conducted with 27 high school marine science teachers and their students in 26 schools in Tampa, Florida. The CHANGE curriculum used a local, place-based approach using scientific data gathered from the Florida Gulf Coast and incorporated activities including (a) a scientifically web-based science fiction novel about future Florida residents, (b) computer games, and (c) hands-on laboratory activities. We collected both quantitative and qualitative data. The quantitative data collected included students’ midterm and final exam scores and surveys about student perceptions of climate change science. Our qualitative data included classroom observations, focus groups, and open-ended questions in the student surveys. On midterm and final exam questions related to climate change science, students who participated in the CHANGE curriculum scored significantly higher than their peers who did not participate in the curriculum.

Instruments: The quantitative data collected included students’ midterm and final exam scores and surveys about student perceptions of climate change science. Our qualitative data included classroom observations, focus groups, and open-ended questions in student surveys.

Key Challenge: The challenge in evaluating this type of large invention (adding a variety of curricular materials to a course) is isolating the effects of individual parts of the intervention. You can see positive impact with midterms and exams across school years. However, how do you know which parts of your materials made the differences? You can deploy surveys, conduct interviews with teachers and students and conduct focus groups with students, but they supply perception of learning data which is not the same as evidence of actual learning, nor do they provide airtight evidence for contributions to learning of the different parts.

Product(s): CHANGE provides a website, https://climatechange.usf.edu/ which includes nine units from a marine sciences course, complete with lesson plans involving inexpensive, easy to find materials, Powerpoints, downloadable files and an interactive web-based eBook with simulation-based games. The nine units for high school-level Marine Science classes include: (1) Ocean Exploration, (2) Marine Geology, (3) Marine Chemistry, (4) Estuaries, (5) Marine Physics, (6) Populations: Producers, (7) Populations: Invertebrates, (8) Populations: Vertebrates and (9) Capstone: Apollo Beach. All of these materials can be potentially repurposed for other high school science courses.

 Teachers can view the top level, outline of the CHANGE curriculum web-page: https://climatechange.usf.edu/. However, to access the actual materials, they will need to register to get a username, by emailing Dr. Glenn Smith: glenns@usf.edu and metinbesalti@mail.usf.edu

Publications

Dobson, A., Feldman, A., Nation, M., and Laux, K. (2019). Red Tide. The Science Teacher 87(1): 35-41.

Smith, G. G., Besalti, M., Nation, M., Feldman, A., Laux, K. (2019). Teaching Climate Change Science to High School Students Using Computer Games in an Intermedia Narrative. Eurasia Journal of Mathematics, Science and Technology Education, 15(6), em1698. https://www.ejmste.com/article/teaching-climate-change-science-to-high-…;

Nation, M., Feldman, A. and Wang, P. (2015). A rising tide. The Science Teacher, 82(6), 34-40

Feldman, A., Nation, M., Smith, G. G., & Besalti, M. (2017). The Use of Complementary Virtual and Real Scientific Models to Engage Students in Inquiry: Teaching and Learning Climate Change Science. In Levin, I., & Tsybulsky, D. (Ed.), Optimizing STEM Education With Advanced ICTs and Simulations (pp. 30-57). IGI Global. https://www.igi-global.com/chapter/the-use-of-complementary-virtual-and…;

Nation, M. and Feldman, A. (in review). Environmental Education in the Secondary Science Classroom: How Teachers’ Beliefs Influence Their Instruction of Climate Change. Journal of Science Teacher Education.

Feldman, A., Nation, M. and Laux, K. (in review). The effects of extended action research-based professional development on the teaching of climate science. Educational Action Research.

Two dissertations:

How Teachers' Beliefs About Climate Change Influence Their Instruction, Student Understanding, and Willingness to Take Action, Molly Nation, June 2017, University of South Florida.

Games for CHANGE: High School Students' Learning Experiences and Motivation to Learn Climate Change Science through Educational Computer Games, Metin Besalti, May 2019, University of South Florida.


Connected Biology: Three-dimensional Learning from Molecules to Populations (Collaborative Research)

(PIs: Frieda Reichsman, Peter White)
Grade Level: 9-12
Target Audience: Biology teachers from Intro to AP

Disciplines/Subject Areas: Evolution, Cell Biology, Population Biology, Protein Synthesis, Proteins, DNA

Project Description: The ConnectedBiology project has developed technology-enhanced lessons for high school biology that foster the integrated learning of genetics and evolution. This exciting set of 15 lessons, "Deer Mouse Fur Color: From the Field to the Beach" connects macromolecules, cell biology, and inheritance of mouse fur color with the evolution of fur colors in a population. Students use a Multi-Level Simulation (MLS) that connects both visible and invisible events into a causative chain across levels—with the ability to “zoom” in and out of the population, organism, cell, and molecular levels.

The lessons are free, research-based, and NGSS-aligned. The curriculum package includes online lessons, an interactive Teacher’s Edition, and a real-time Teacher Dashboard. Additional background materials and supplemental resources are also provided. Connected Biology is a collaborative effort between Michigan State University and the Concord Consortium.

Describe the data that students work with in your project:  1. Data produced by students via various embedded simulations and 2. Data collected professionally and published by scientists researching the evolution of deer mouse fur color.

Initial Findings Related to Analyzing & Interpreting Data: In Unit End Summary Discussions, students are demonstrating that they understood the lessons and materials and can explain how the given component (e.g. predation, protein synthesis) influences the fur color of a deer mouse. Furthermore, students are showing that they are starting to make connections across lessons (and therefore levels of scale). For example, students can explain that a single nucleotide change in the DNA sequence will cause not only a change in protein structure, but will also influence the protein function, bridging the molecular and cellular levels. They are also able to explain that the genetic material inherited by a mouse will influence its ability to produce pigment-producing eumelanin. Finally, students appear to be engaged and excited about the lessons. From classroom observations, we’ve seen that students stay on-task when working on the lessons, and are animated in their speech when discussing the material. For example, one student even used her hair to model to a neighbor how the chromosomes needed to condense into chromatin before cell division in order to prevent chaos. As another example, students seem to be connecting to the material. Students use phrases like “the hawk is gonna getcha,” or “should I get a pet mouse,” which demonstrates to us that students find the material relatable.

Instruments:  We designed a pre- and post-survey to collect student conceptions of their knowledge and interest both before and after completing the Deer Mouse Case.

We have embedded questions in the material that are used to assess student knowledge at a given point.

We use code audio and written responses to the summary questions for each unit.

Key Challenge: The schools and students using our materials in the pilot years are all unique. As with all education research, and especially qualitative education research, it can be tough to separate out what was learned specifically as a result of our materials from teacher and school impact.

Standardized testing greatly impacts what teachers can do and how they can apportion time on topic in the classroom. It is difficult for us to get wide implementation when the materials go sufficiently deep to explore a phenomenon with three-dimensional teaching and learning. Being more flexible about how teachers implement is our strategy, but it is not always satisfying if the implementation changes too much from the intended use of the materials.

Product(s):  Curriculum | Website

Videos:

Publications:

Kolonich, A., Warwick, A., Mead, L., Reichsman, F., Horwitz, P., White, P. J. T., Smith, J., McElroy-Brown, K. (2019) “Using high school students’ initial perceptions of evolution across biological levels to inform curriculum development.” National Association for Research in Science Teaching, Atlanta, GA, USA.

Warwick, Alexa R., Kolonich, A., Bass, K. M., Mead, L. S., Reichsman, F. (Accepted for publication) "Ten Simple Rules for Partnering with K-12 Teachers to Support Broader Impact Goals" PLOS Computational Biology.

Ellis, R., Reichsman, F., Mead, L.S., Smith J., McElroy-Brown, K., White, P.J.T. (Submitted) ConnectedBio: An Integrative and Technology-Enhanced Approach to Evolution Education for High School. American Biology Teacher

Ellis, R., Mead, L.S., Reichsman, F.,Smith J., McElroy-Brown, K., White, P.J.T. (In preparation) Connected Biology: Student ability to connect biological processes across scales.


Crowdsourcing Neuroscience: An Interactive Cloud-based Citizen Science Platform for High School Students, Teachers, and Researchers

(PI: Camillia Matuk; Co-PI: Ido Davidesco; NYU Project Lead: Suzanne Dikker)
Grades: 9-12
Target audience: High school students and teachers, human brain and behavior scientists.

Disciplines/Subject Areas: Students are learning about the process of generating scientific knowledge by engaging in practices such as developing research questions, analyzing existing scientific research data, and proposing, designing and critiquing research studies. Through these research activities, they are also learning about human brain and behavior science.

Project Description: MindHive is a digital citizen science platform focusing on human brain and behavior research. MindHive supports virtual Student-Teacher-Scientist (STS) partnerships using an open-science approach: Students, teachers, and scientists across the globe are invited to contribute experiments, resources, and research data to the platform, thus supporting both STEM learning and scientific discovery. By involving young citizen scientists in each other’s projects through peer-review and data collection, students will learn that scientific progress is a collaborative, iterative, and transparent process.

Examples of ongoing MindHive projects include research focusing on the psychological effects of the coronavirus pandemic on high school students (co-designed with a group of high school students); stress and coping in low-SES neighborhoods in New York City; and the role of social influence to explain whether adolescents engage in environmentally conscious behavior.

Describe the data that students work with in your project: Data include participants' responses to online tasks designed to answer questions about human cognition and social psychology. The studies are hosted and run by the online platform, MindHive, and are designed by human brain and behavior scientists and high school students.

Initial Findings Related to Analyzing & Interpreting Data: We conducted a pilot study in the Spring of 2020 with 17 students in an environmental science class at a private high school. In this fully online unit, we guided students in first participating in scientist-designed research studies on brain and behavior, then proposing their own studies. While the time boundaries of the pilot did not allow for students to collect data, they engaged with data in important and formative ways. For example, students participated in research studies with their classmates and examined and discussed patterns in their shared class data. They also used other existing research data to inform their study proposals.

Our early findings indicate that students found the experience of proposing their own research projects, doing background research, and learning about the scientific research process through scientific pre-prints and the peer review process, to be very insightful. Participating in the citizen science curriculum—particularly their participation in brain and behavior experiments—made the scientific process more transparent to them, and revealed the collaborative, complex, and iterative nature of scientific research. Students were excited about moving onto the data collection process.

Instruments: We piloted pre- and post-test assessments that collected information on students’ attitudes, behaviors, and identities related to science.

Key Challenge: One major challenge in implementing this project has been in devising ways to orchestrate interactions between scientist mentors, teachers, and students, such as to maximize the benefits to each group while being sensitive to their other professional needs and commitments. On the one hand, we want to give students agency to pursue their own research questions and study designs; on the other, we want to ensure that students’ projects generate sufficient data to be useful to their scientist partners.

Product(s): Products from this project will include curriculum modules, a browser-based platform for designing and running research studies (mindhive.science), and videos produced by scientists to describe their research. Products will also include publications that share our research findings on how to best support student-teacher-scientist partnerships, how to implement citizen science in classroom settings, and on the impacts of citizen science on students' and teachers' identities and attitudes toward, and understanding of the scientific process. All products will be shared on the MindHive website:https://wp.nyu.edu/mindhive/


EcoXPT: Learning about Ecosystems Science and Complex Causality through Experimentation in a Virtual World

(PI: Tina Grotzer)
Grades: 6-8
Target Audience: All Learners

Disciplines/Subject Areas: Ecosystems/Environmental Science (see NGSS alignment)

Project Description: EcoXPT is a three-week middle school curriculum focused on how ecosystems work. Students explore an immersive simulation of a pond ecosystem traveling between different days and locations. Eventually, they discover an environmental puzzle that they attempt to explain by using virtual tools, observing organisms in the world, and by collecting and graphing data. For instance, they can take measurements of water quality, temperature, and population levels of microscopic and macroscopic organisms. Students collect data, graph their findings, and learn from a variety of representations that help them to reason about variables, visible or not, in the world. Students meet virtual ecosystem scientists and use authentic modes of ecosystems experimentation—both in a virtual lab and out in the virtual world. They use an on-line concept map to make claims, collect evidence, and to offer reasoning in support of their claims as they develop explanations.

Describe the data that students work with in your project: Students collect data in a virtual world using virtual measurement and monitoring tools. The data is realistic—developed from authentic contexts with our collaborating ecosystems scientists—however, it does not have the same level of variability as real-world data. Students collect the data and populate a data chart that allows them to explore the data. Variable include temperature, phosphates, nitrates, dissolved oxygen, bacteria, algae, fish populations, etc.

Initial Findings Related to Analyzing & Interpreting Data:  Studies of EcoXPT in the classroom revealed the following findings. In a study contrasting the performance of students using EcoXPT with the experimental tools that ecosystems scientists use to students using a version that omitted the experimental tools, both groups made significant gains in understanding ecosystems content, dynamics across space and time, causality, experimental methods, and attitudes towards science. However, those with the experimental tools demonstrated significantly greater understanding of experimental methods and the differences between correlation and causation. Tested against a paper-based curriculum focused on a similar ecological puzzle, EcoXPT students made significant gains in attitudes towards science, experimental methods, and causality, while students using the paper-based version made significant gains only in attitudes towards science. Case studies contrasting EcoXPT to other technology-based programs suggest that EcoXPT is especially helpful in supporting students’ understanding of the complex dynamics of ecosystems. A case study testing a focus on developing strong body of evidence for explanations led to students initially making fewer connections but having deep and varied support for each connection that they made. The work also resulted in new methodologies for coding how well teachers implement inquiry-based learning curriculum; this should benefit other researchers. EcoXPT has already positively impacted the learning of over 2,000 students who participated in the development and testing phases of the work and has impacted the practice of 20 teachers.

Instruments: We used assessments that were modified from earlier work of the Project Team. The instruments were modified to reflect how Ecosystems Scientists frame experimentation (as we researched and published in Kamarainen & Grotzer, 2019 in BioScience) and scientific explanation. The instruments focused on assessing complex causal understanding and how ecosystems scientists engage in explanation when Control of Variables forms of experimentation are not possible.  There were no existing instruments which is why we developed our own or modified earlier instruments from our Causal Learning in the Classroom Project (CLiC). We also developed a method to consider fidelity of teaching an inquiry-based immersive simulation. These instruments are forthcoming in our publications.

Key Challenge: It is challenging to assess the differences between how students think about correlation versus causation in their written explanations and concept maps because they are typically happy to confound the two--using correlational patterns as causal without investigating whether they actually are causal.  Concept maps and/or written explanations do not easily offer a way to tease these understandings apart. We developed a set of questions focused on what conclusions are warranted from given information to discern these understandings. We also developed a means to code for the sources of evidence in students’ responses with a particular focus on mechanism information.

Product(s): The full curriculum includes lesson plans, PowerPoints, a set of six Thinking Moves (aligned to the NGSS) with supporting videos, an extensive teachers’ guide, and two sets of Professional Development workshop sessions.  EcoXPT is available for free download and use by classrooms around the world at this link:https://ecolearn.gse.harvard.edu/projects/ecoxpt

EcoXPT was funded by NSF grant DRL-1416781 to Tina Grotzer and Chris Dede, and builds on prior research with EcoMUVE, supported by IES grant R305A080514.


GeoHazard: Modeling Natural Hazards and Assessing Risks

(PI: Amy Pallant)
Grades: 5-12
Target Audience: Earth Science and Environmental Science teachers

Disciplines/Subject Areas: Earth Science and Environmental Science (see NGSS alignment)

Project Description: The GeoHazard: Modeling Natural Hazards and Assessing Risks project gives students the opportunity to experiment with powerful, data-driven Earth system models to investigate fundamental science concepts surrounding natural hazards, risk, and impact. Students use evidence from their explorations to develop scientific arguments focused on risk analysis and the level of uncertainty related to risk. Our first module focuses on hurricanes. While hurricanes are a natural part of our climate system, increasingly destructive hurricanes put a growing number of people and structures at risk. The Hurricane Module has been designed to reveal important concepts necessary for students to develop a deep understanding of what factors drive hurricane movement and intensity especially as global temperatures continue to rise. As students explore these variables through our interactive Hurricane Explorer model and real-world case studies, they also consider how hurricanes impact people and their communities.

Describe the data that students work with in your project: As part of the GeoHazard project, students work with data sources generated from 1) experimentation with computational models and 2) professionally generated data. We are developing three modules: Hurricanes, Wildfires and Floods. Data sources for the Hurricane Module, which is currently freely available for public use, are described below.

Model data: The Geohazard project is creating computational models and simulations that students will use to explore complex Earth systems that govern the progression of wildfires, hurricanes and floods. The Hurricane Explorer, for example, allows students to set the location and magnitude of pressure systems and adjust the sea surface temperature in order to simulate hurricane tracks. Students use data produced by the model to investigate hurricane movement and intensity as well as hazards related to precipitation, wind speed, and storm surge...Read more

Initial Findings Related to Analyzing & Interpreting Data: Using a NOAA produced forecasting map with an actual hurricane with the cone of uncertainty, students were asked about future locations and strengths of the hurricane before and after the module. We used the true/false item format to create six items. Overall, half of the students had difficulties interpreting the cone of uncertainty prior to the module: 56 % students thought that the cone of uncertainty includes all possible future locations; 51% thought that the hurricane would follow the middle path inside the cone; 62% thought that the hurricane would grow to fill in the entire cone over time. However, it appeared easier for students to recognize that (1) predicting further into the future was more difficult than closer to the future (67%); (2) people outside of the cone should also be concerned (88%); and (3) predicted changes in the hurricane strength could be identified from the map (78%). On these six items, Students made significant gains from pretest (M = 3.64) to posttest (M = 4.09) with the effect size (Cohen’s d) of 0.34, t (192) = 4.69, p < 0.001...Read more

Instruments: We have developed a construct that represents what students need to know about natural hazards, risks, and impacts. We assumed a common underlying construct responsible for student performances on pretest, posttest, and embedded assessments. The construct consists of disciplinary ideas related to (1) identifying the difference between a natural phenomenon and a hazard, (2) describing how hazards impact human lives, health property, (3) elaborating how a range of factors affect hazard formation and progression, (4) forecasting risks and uncertainty involved, (5) considering ways in which humans can mitigate risk, (6) explaining the role climate change might play in future risks and impacts caused by hazards. Our hurricane hazard instrument consisted of 29 items: 13 multiple choice items, 6 true/false items, and 10 open-ended explanation items. All but one multiple choice and true/false items were scored 0 (incorrect) or 1 (correct); Open-ended explanation items were scored from 0 to 4 according to the knowledge integration scale: 0 (off-task, I don’t know), 1(incorrect ideas or links), 2 (scientifically valid ideas), 3 (a link between two scientifically valid ideas), and 4 (two or more links among three or more scientifically valid ideas). A total of 60 points were possible. The reliability among the items was 0.78 using Cronbach’s alpha.

Key Challenge: Our current challenge is to run continued classroom tests on the Hurricane Module as well as on our two newer modules that focus on wildfires and floods. Due to the unprecedented school closures and move to remote teaching and learning, we will be experimenting with remote focus groups, classroom observation and data collection. We have already completed the development of robust online teacher training materials, which are freely available for the Hurricane Module. We have a cohort of teachers who have agreed to pilot test the Wildfire Module and Flood Module this year in whatever classroom setting they may be in.

Product(s): Website | Hurricane Curriculum


Integrating Chemistry and Earth science (ICE)

(PI: Alan Berkowitz)
Grade Level: 9-12
Target Audience: Our target audiences are High School Chemistry students, their teachers, and more broadly, the school districts who offer this kind of infused Earth-chemistry course. In Baltimore City (City Schools), this course is most often taught in the sophomore or junior year of high school.

Disciplines/Subject Areas: Our work addresses Chemistry and Earth Science disciplinary content. Specific content includes:

  • heat energy pathways, processes and budgets; urban heat island;
  • mountain and rock formation; weathering, erosion and deposition; chemistry of urban watersheds and streams

Our core ICE units address all 8 NGSS practices with specific emphasis on developing and using models, analyzing and interpreting data and constructing explanations.

Project Description: With the adoption of NGSS by the State of Maryland in 2013, all local districts were challenged to ensure their students received adequate instruction in three science areas: biology, physical science, and Earth science.  Baltimore City Public Schools adopted an Earth science infusion approach whereby those standards would be taught in the courses of the three existing science courses (biology, chemistry, physics).  City Schools partnered with the Cary Institute of Ecosystem Studies and George Washington University to work specifically on modifying the high school chemistry course. The ICE project addresses the following goals:

  • Develop 3-D Earth science infused chemistry units and subsequent assessments that bring together lab and field investigations and exploration of local data sets to support teaching and learning about compelling phenomena in the Baltimore environment.
  • Provide professional development, in-school support and resources for district-wide adoption.
  • Collect, analyze and present evidence to address hypotheses about learning and teaching.

Describe the data that students work with in your project: Students work with either second-hand data collected by scientists, or data they collect themselves. Second-hand data sources include the Baltimore Ecosystem Study, Princeton University and others. Two examples from the curriculum include:

Examples of first-hand data and associated activities include:

  • Physical Weathering- Students use a rock tumbler to investigate how susceptible urban building materials are to weathering and erosion.
  • Chemical Weathering- Students complete a laboratory simulation of the effect of acid rain (vinegar) on sedimentary rock (chalk).

Initial Findings Related to Analyzing & Interpreting Data: A key student generated artifact produced at the conclusion of the data-focused, urban heat island (UHI) lesson sequence is a model-based explanation of the UHI phenomenon. In this modeling activity, students are tasked with describing the effect of various building materials and aspects of a cityscape that contribute to the UHI effect in their hometown. In these models, students must demonstrate how energy moves into, within, and out of the cityscape system. In analyzing these artifacts we rely on a three part researcher-developed, model-based explanation rubric (Zangori, et al., 2017) that assesses: (a) students’ identification of relevant system components; (b) sequences or cause-effect relationships contributing to the phenomenon; and, (c) explanations that link sequences to account for the phenomenon. Our initial findings suggest that the lesson sequence supports students to identify relevant components of the UHI phenomena (avg: 2.58/3 poss. pts.), but demonstrated less proficiency in identifying sequences within the phenomenon (avg: 1.42/3 poss. pts.) or explanatory mechanisms at play (0.44/3 poss. pts.). Additionally, students tend to rely on general qualitative descriptions of properties or characteristics of materials represented within their models. Therefore, in subsequent curricular revisions, it is necessary to further support teachers to help students connect the empirical aspects and authentic data generated during the lesson sequence to the model building activities.

Instruments: To date, our research team has generated a series of assessment items and item clusters to elicit students’ abilities to analyze data and generate arguments in a claim-evidence-reasoning format. These items are still undergoing refinement and pilot testing. Additionally, we have developed the modeling task used in the UHI lesson sequence as well as a modeling task connected to an erosion and weathering lesson sequence within the curriculum. Associated with these two tasks, we have generated task-specific model-based explanation rubrics to assess students’ use of components, sequences, and explanations within their models. Within the research team, we have demonstrated consistent inter-rater reliability when applying these rubrics to student generated models. 

Key Challenge: Like many current projects, our work has been directly impacted by the ongoing COVID-19 pandemic. While there are aspects of the ICE curriculum distributed across the school year, a primary area of focus for curricular integration and data collection comes toward the end of the school year. This structural aspect of the curriculum makes the ICE project particularly sensitive to typical challenges within the school year (e.g. snow days, mandated testing) that impact a teachers’ ability to complete all planned activities, which was further exacerbated by the move to virtual instruction during the later portion of the recent school year in response to the pandemic. Throughout the course of the project, the curriculum has undergone several iterations each year resulting in refined timing and pacing to support successful implementation. Serendipitously, the move to virtual instruction during the recent school year provided an opportunity for the research and development team to reflect on the critical aspects of the curriculum essential for meeting our learning goals. These insights gained have helped guide the most recent refinement of the curriculum.

Product(s): Our ICE team has infused Earth science into two of the seven units in the Baltimore City Schools’ Chemistry curriculum: Thermochemistry and the Chemistry and Baltimore’s Mountain. One example of our curriculum is a lesson sequence that supports students to move through a series of activities that build foundational knowledge for explaining the urban heat island effect. Students collect data in the school yard by measuring the surface temperatures of common materials found in and around their school, e.g. brick, concrete, asphalt, grass, dirt, shaded and unshaded surfaces, etc. Then they design an investigation to generate data related to the heat capacity of common building materials found in the city, e.g. brick, concrete, granite, marble. Eventually students explore a published dataset with day and nighttime temperatures collected at urban and suburban sites in Baltimore during a recent heat wave. Students analyze these data to build on previous experiences and move toward generating an explanatory model of the urban heat island effect that occurs in their hometown and is a common phenomenon across urban locations.


InquirySpace 2: Broadening Access to Integrated Science Practices

(PI: Chad Dorsey)
Grades: 9-12
Target Audience: High school students who are taking physics and biology courses with labs

Disciplines/Subject Areas: Physics and Biology (see NGSS alignment)

Project Description: Every student should have the chance to experience the exciting practice of science. Instead, far too often, students encounter only highly structured and constrained “cookbook” labs in their science classrooms. InquirySpace combines a software environment that integrates probeware and data exploration capabilities with instructional guidance and scaffolds students’ transition from fundamental data analysis and guided experiments to open experiments of their own design. We use innovative technologies—the versatile modeling environment of the Molecular Workbench, real-time data collection from probes and sensors, and the powerful visual data exploration capabilities of our Common Online Data Analysis Platform (CODAP). These tools are integrated into a coherent, online environment enabling rich, collaborative scientific inquiry. Student materials are designed to help students experience open-ended investigations through a sequence of activities with increasing degrees of student autonomy, culminating in an open-ended exploration of their own design.

Describe the data that students work with in your project: Students design and create setups using physical materials available to them or in the science lab. Students decide what data to collect and use sensors to gather measurements. Investigations include fading scaffolds of guidance and feedback related to data collection. Students first observe sensor data outputs to make modifications to their setups. Once students are familiar with setups and data displays on the computer screen, they begin formal data collection. Using the sensor output on the screen, students analyze outliers and select data segments. Sometimes students need to numerically transform data, e.g. position to velocity to acceleration, to create data visualizations such as graphs. Students then identify patterns between independent and dependent variables from the graphs and use data as evidence to write explanations about the relationships among variables. In some cases, students work with data generated from simulations about physical phenomena addressed in the module and conduct similar analyses with more data.

Initial Findings Related to Analyzing & Interpreting Data: The IS2 physics module is designed to engage students in inquiry-oriented, data-intensive, sensor-based scientific experimentation. In particular, the practices of “planning and carrying out investigations” and “analyzing and interpreting data” are the main instructional foci. In the past year, five high school physics teachers implemented the module on their own with minimal support from project staff. We analyzed students’ pretest-posttest responses for evidence of these two practices. Students (N = 145) of five teachers took both the pretest and posttest. Ninth grade students from three teachers made 1.2 standard deviation gains (p < .001), while students in a teacher’s 12th grade AP class made 0.98 standard deviation gains (p < .001)...Read more

Instruments: We developed instruments that measure students’ abilities to practice “planning and carrying out investigations” and “analyzing and interpreting data” in the context of scientific experimentation in physics and biology. We applied a construct-modeling approach (Wilson, 2004) to theorize the construct boundaries and sophistication levels, develop items and scoring rubrics, and analyze and interpret results. The major reason for developing new instruments is to comprehensively represent a variety of reasoning needed to carry out data-based experiments such as designing experiments, enacting control of variables, identifying and addressing systematic and random errors, statistically treating data points, creating data representations, and recognizing data patterns. Currently, this array of reasoning is scattered across literature. As we put these different types of reasoning under the umbrella of “planning and carrying out investigations” and “analyzing and interpreting data” practices, our psychometric analyses will test, once a sufficient number of students have taken the instruments, whether uni-dimensionality of the construct can be justified, and if so how these different reasoning types compare on the unified scale...Read more.

Key Challenge: In general, the teachers we approach express strong interest in engaging their students in inquiry with real world data. The roadblocks to their participation often stem from external pressures to cover content at a rapid pace to meet district requirements and prepare for standardized tests. A challenge for us is to help them develop strategies that can enable them to implement investigation of noisy, real world data in a limited time frame.

Product(s):

Webpages and Platforms:

  • The InquirySpace Collections webpage offers an introductory video and teacher resources
  • The InquirySpace webpage provides information about the IS2 project, including video, project description, learning goals, module access, resources, and publications
  • The CODAP website (Common Online Data Analysis Platform) is an open-source software for dynamic data exploration
Teacher Guides and Online Activities:

Physics Module
Investigation 1 -How would a scientist study a phenomenon like bungee jumping? (Introduction to experimentation and data collection/analysis) Teacher Guide | Online Activity
Investigation 2 -How do you measure how fast something is going? (Velocity) Teacher Guide | Online Activity
Investigation 3 -What makes an amusement park exciting? (Acceleration) Teacher Guide | Online Activity

Biology Module
Investigation 1 -Why does fresh pineapple prevent jello from solidifying? (Enzymes) Teacher Guide | Online Activity
Investigation 2 -How do our cells keep just the right balance, letting good stuff in and bad stuff out? (Diffusion and osmosis) Teacher Guide | Online Activity
Investigation 3 - Is my salad alive? (Photosynthesis and cellular respiration) Teacher Guide | Online Activity

Publications & Articles:

Haavind, S. & Murtha, M. (2020). Accessible physics for all: Providing equity of access for high school physics with extended experimentation and data analysis. The Science Teacher. July/August, 2020 pp. 54-58.

Sarah HaavindInnovator Interview(Spring 2020). @Concord

Damelin, D., Lee, H. -S. & Stephens, L. (Fall 2018).Inquiry Space Model of Scientific Experimentation.  @Concord

Hee-Sun LeeInnovator Interview (Spring 2018). @Concord

Blogs:


Schoolyard Science Investigations by Teachers, Extension Volunteers and Students (Schoolyard SITES)

(PI: Lara Gengarelly)
Grades: 2-5
Target Audience: Elementary teachers (2nd-5th grade) and University of New Hampshire Cooperative Extension science volunteers

Disciplines/Subject Areas: Life and Earth Science

Project Description: The Schoolyard SITES program partners elementary teachers with University of New Hampshire Cooperative Extension science volunteers to bring locally-relevant citizen science projects to elementary students and to increase teachers’ self-efficacy teaching science. With support from science volunteers, teachers develop science investigations that incorporate student learning goals aligned with the Next Generation Science Standards and involve existing citizen science projects. Each teacher-volunteer team designs and teaches a science project that is relevant to the school district’s curriculum and school site. Schoolyard SITES students engage in real-world, problem-based learning and investigate their schoolyard using the scientific process, and they contribute valuable scientific data to a variety of existing citizen science initiatives, ranging from Project FeederWatch to local maple sap monitoring.

Describe the data that students work with in your project: The data that the elementary students work with in the Schoolyard SITES program is student-collected. Each teacher-volunteer team and class determine the particular type of data collected based on the driving scientific question and available citizen science protocols.

Initial Findings Related to Analyzing & Interpreting Data: Schoolyard SITES is a University of New Hampshire research study that investigates a new professional development model for elementary school teachers. The program partners teachers with Cooperative Extension science volunteers to create a community-based professional development partnership that improves educators’ use of locally-relevant, citizen science projects in the classroom. The model builds on the premise that both groups have expertise that can be shared and collaboratively developed.

According to initial research results, all cohort I Schoolyard SITES teachers gained new science content and improved their integration of the NGSS science practices in their classroom teaching. Analyzing and interpreting data was one of the top NGSS science practices teachers gained confidence in teaching along with students’ carrying out investigations, asking scientific questions, and designing and using models.
 

Instruments: What existing or project-developed instruments are you using to measure students’ knowledge or skills related to working with data? For project-developed instruments, what led you to adapt or create new measures, and what psychometric information (if any) are you able to share at this time?

Key Challenge: While citizen science projects are an opportunity for students to work with data, one of the challenges for the teacher-volunteer teams has been determining how to incorporate existing citizen science initiatives to support their main science learning goals. As a response to this challenge we created a project guide for teacher-volunteer teams to use when designing the Schoolyard SITES projects. The guide helps the teams intentionally frame the classroom projects based on learning goals and NGSS science practices (i.e., data analysis and interpretation) as well as identify appropriate connections to existing citizen science programs (e.g., Project FeederWatch).

Product(s): Website | Video


Scientific Data in Schools: Measuring the Efficacy of an Innovative Approach to Integrating Quantitative Reasoning in Secondary Science (Collaborative Research)

(PIs: Molly StuhlsatzLouise Mead)
Grades: 9-12
Target Audience: While the primary users are middle and high school students, Data Nugget activities are also available for upper elementary and undergraduate students.

Disciplines/Subject Areas: Data Nuggets are typically written by scientists about their own research. Content represents contemporary science and emerging ideas in the fields of biology, ecology, evolution, environmental science, physics, human biology, forensic science, agricultural science, chemistry, and astrobiology. Data Nuggets follow the process of science taken by the researcher, which gives students the opportunity to repeatedly engage in analyzing and interpreting data, mathematical thinking, constructing explanations using evidence, and argumentation (NGSS SEPs 4-7).

Project Description: Data Nuggets (http://datanuggets.org) are free classroom activities, designed to engage K-16 students in the practice of science by providing them with authentic data collected by scientists. In each activity, students read background information on a study system and scientist, graph and interpret authentic data from the scientist’s research, and use graphs to construct explanations based on sound reasoning and evidence. By relying on authentic research and data, Data Nuggets’ innovative approach reveals the process of science, while building student quantitative abilities and interest in science. The inclusion of the story behind the research engages students in the journey taken by the scientist as they formulated their research questions and ideas.

Describe the data that students work with in your project: Each Data Nugget includes authentic data that highlights research from scientists at all stages in a scientific career. We explicitly focus on real, messy data, including data that do not support the hypothesis, show no significant trend, and have limitations in interpretation based on collection techniques. In this way, Data Nuggets provide students with a glimpse into what science really looks like, creating opportunities to think deeply about how data can be used to inform scientific results and explore the limitations of data.

Initial Findings Related to Analyzing & Interpreting Data: In the efficacy trial, students in classrooms assigned to the Data Nuggets treatment scored, on average, about the same as students in the comparison condition on the quantitative reasoning assessment. However, within the treatment group, teacher implementation was crucial. Classrooms where the strongest implementation was observed significantly outperformed those with the poorest implementation. We also found that students using Data Nuggets were significantly better at developing a data-based explanation. Further, students in the Data Nuggets treatment were more likely to report higher levels of self-efficacy around analyzing and interpreting data and higher motivation to pursue a science career.

Instruments: Our goal was to identify and create instruments to measure students’ ability to interpret representations of data in the context of biological phenomena. We also wanted to assess students’ development of graphical representations of data and ability to translate that graphical representation into a written explanation. We determined that there was not an instrument available that met our needs, so we developed an assessment. The assessment focused on quantitative reasoning and includes data about variation within and between groups, time series data, covariational data, and a graphing and explanation task where students use data we provide them to develop a graph and then respond to a series of interpretation prompts (Rasch person reliability for common item equated measure = 0.80, person separation = 2.02; Cronbach’s α = 0.80).

Key Challenge: While Data Nuggets are standalone activities easily integrated into curriculum, we are now exploring the importance of implementation. Rather than handing out a Data Nugget to be done as a worksheet, a strong implementation involves careful consideration of scaffolding opportunities, making connections to course content, understanding students’ starting points and misconceptions, and engaging students in discussion to push their thinking deeper. Data Nuggets teacher guides are full of recommendations for discussion topics, including specific areas to check student comprehension of content, research design, data variables, and interpretation of datasets. We are also working on videos to showcase strong implementation cases.

Product(s):

  • All Data Nuggets can be found at datanuggets.org.
  • Slides from our professional development offerings, including an introduction to Data Nuggets and training in quantitative reasoning, process of science, identifying appropriate hypotheses, graphing and data exploration, constructing explanations, and asking strong scientific questions:http://datanuggets.org/study/pd/

Strengthening Data Literacy across the Curriculum

(PI: Josephine Louie; Co-PIs: Beth Chance, Soma Roy)
Grades: High school, primarily Grade 12
Target audience: Students taking non-AP mathematics and statistics courses. We seek to reach Black, Latinx, and low-income student populations – groups that have been historically underrepresented in STEM and who are currently underrepresented in emerging data science fields.

Disciplines/Subject Areas: Statistical reasoning, the four-step data inquiry cycle, and social studies content (focusing on income inequality and immigration in the U.S.).

Project Description: The Strengthening Data Literacy across the Curriculum (SDLC) project is developing and studying prototype high school curriculum modules that integrate social justice topics with statistical data investigations to promote skills and interest in data science among underrepresented groups in STEM. Iteratively developed and tested in collaboration with high school statistics and social studies teachers, the modules consist of extended sets of applied data investigations in which students explore patterns of social and economic inequality in the U.S., using large-scale data sets from the U.S. Census Bureau and the data visualization tool CODAP. The project hypothesizes that high school students – particularly those from historically marginalized groups – may develop deeper understandings of data concepts and practices as well as greater interest in statistics and data analysis when they can use data to examine questions of social justice that are relevant to their own lives.

Describe the data that students work with in your project:Students analyze data from the annual American Community Survey and the U.S. decennial census, both collected by the U.S. Census Bureau.

Initial Findings Related to Analyzing & Interpreting DataBased on results from pre- and post-tests administered to students (n=210) who completed one of the three-week SDLC modules in fall 2019, students demonstrated  statistically significant growth in individual interest in statistics and working with data, with a small effect size (d=0.25). Preliminary analyses also show that students displayed statistically significant gains in understanding of core statistical concepts, particularly in measures of center as well as in multivariable reasoning.

Instruments: The project is measuring student interests in statistics and data analysis using scales that have been adapted from instruments published by Linnenbrink-Garcia et al. (2010) and Sproesser, Engel, & Kuntze (2016). Reliability statistics for adapted situational interest, individual interest, and self-concept scales, as measured by Cronbach alpha statistics, range from 0.73 to 0.92. 

The project is also measuring students’ understanding of core statistical concepts using items from the Levels of Conceptual Understanding of Statistics (LOCUS) instrument (Jacobbe, Case, Whitaker, & Foti, 2014), and items from the Comprehensive Assessment of Outcomes for a First Course in Statistics (CAOS) (Garfield, delMas, Chance, & Ooms, 2006).

Key Challenge:One challenge is supporting discussions of social justice in mathematics/statistics classes, when neither teachers nor students may be comfortable with such discussions in these settings. Module resources in this domain are a work in progress.

Another challenge is achieving the right mix of student scaffolding, structured activities, and open data exploration in the modules. Although more open data investigations have the potential to be highly engaging for students, they require strong facilitation by teachers to ensure student learning. SDLC modules have structured lessons followed by a more open team data investigation, but this approach is a work in progress.

Product(s): Curriculum modules for high school non-AP mathematics/statistics classes. Each module takes approximately 3 weeks to implement in full. We are also exploring the feasibility of implementing the modules in whole or in part in high school social studies classes.

Sample lessons:

Presentations: Louie, J., Chance, B., Roy, S., Fagan, E., Stiles, J., and Finzer, W. (2020). “Building Statistical Thinking with Social Justice Investigations and Social Science Data.” Virtual poster presentation at the Society for Research on Educational Effectiveness (SREE), March 12, 2020. 

Chance, B., Louie, J., Roy, S., Fagan, E., Stiles, J., Finzer, W. (2020). Building statistical and multivariable thinking with social justice investigations. Presented at the Joint Statistical Meetings Virtual Conference.


Teaching Environmental Sustainability: Model My Watershed (Collaborative Research)

(PIs: Steve Kerlin, Nanette Marcum-Dietrich, Carolyn Staudt)
Grades: 5-12
Target Audience:  Middle and High School Science Students

Disciplines/Subject Areas: Earth and Environmental Science

Project Description: The Teaching Environmental Sustainability: Model My Watershed team includes Stroud Water Research Center, Millersville University, and Concord Consortium. Together, we’re teaching a systems approach to problem solving through modeling and hands-on activities based on local watershed data and issues. Middle and high school students act in their communities while engaging in solving problems they find interesting. Our goal is to promote geospatial literacy and systems thinking by providing students and teachers with access to scientifically valid and easy-to-use watershed tools to accurately examine their own neighborhoods, to define local environmental problems or challenges, and to develop solutions to improve their environment. Instructional materials include a new GIS-based web application called Model My Watershed to analyze real data on environmental impacts related to land use, water quantity/quality, and local socioeconomic impacts. The curriculum provides a cloud-based learning and analysis portal accessible from a Web browser.

Describe the data that students work with in your project: The online GIS-based Model My Watershed®(MMW) Site Storm Model (https://modelmywatershed.org/)  is a component of the WikiWatershed® Toolkit(https://wikiwatershed.org/), a suite of web browser based tools designed to help citizens, conservation practitioners, municipal decision-makers, researchers, educators, and students advance their knowledge and stewardship of fresh water. The Site Storm Model simulates storm runoff and water quality by applying the TR-55 & STEP-L water quality models for a single 24-hour rain storm over a selected land area within the continental United States. The results are calculated based on actual land cover data (from the USGS National Land Cover Database 2011, NLCD2011) and actual soil data (from the USDA Gridded Soil Survey Geographic Database, gSSURGO) for the selected land area of interest...Read more

Initial Findings Related to Analyzing & Interpreting Data: Using students’ local watersheds as the context of exploration and learning, the Teaching Environmental Sustainability - Model My Watershed (TES-MMW) curriculum connects “school science” content to real-world applications by using scientific data, watershed knowledge, and Resource Models and Tool’s (RMT) to provide students with the ability to participate in evidence-based decision-making about issues impacting their local watershed(Gunckel et al., 2012; Kali et al., 2003; Mohan et al., 2009; Orion &amp; Ault, 2007). The study results provide evidence that the TES-MMW curriculum and RMT’s are effective tools for increasing students’ abilities in two of the environmental stewardship characteristics. Students’ watershed content knowledge and knowledge of action increased while their locus of control and intent to act remained statistically unchanged. Increasing students’ watershed content knowledge and their knowledge of actions are important curriculum outcomes. While the study did not find statistically significant changes in students’ intent to act, 33% of students reported that following completion of the TES MMW curriculum, they completed a personal watershed action, or they presented a detailed plan to improve their local watershed. While this number is not statistically significant, it is nevertheless note-worthy because the curriculum did not include a specific lesson that guided students through designing a plan of action or a requirement for them to complete an action (the only part of the curriculum that prompts students to take action is a brief description in an optional further investigation or final project for unit assessment at the end of the last lesson)...Read more

Instruments: The instruments used in this study include: a 15-question watershed content knowledge assessment developed by the Zint and Kraemer, portions of the NOAA B-WET Secondary Science Self Report (Knowledge of Actions, Intention to Act, and Locus of Control), and a semi-structured critical incident technique (CIT) interview protocol to assess students’ engagement and action.

Zint, M. and A. Kraemer. 2012. NOAA B-WET Evaluation System Plan: Student Item Bank. Bay Watershed Education and Training Program, National Oceanic and Atmospheric Administration, Washington, D.C.

 Key Challenge: Our goal is to promote geospatial literacy and systems thinking by providing students and teachers with access to scientifically-valid and easy-to-use watershed tools to accurately examine their own neighborhoods, to define local environmental problems or challenges, and to develop solutions to improve their environment. A challenge we experienced in our project was in designing these data-rich Resources, Models and Tools to universally function on the diversity of devices used in schools ranging from laptops, to chromebooks, to tablets. Our project addressed this challenge by creating web-based tools and testing these web-based applications on a diversity of devices and web browsers.

Product(s): Website  

Curriculum: The curriculum unit was designed for students to learn systems thinking and geospatial analysis skills in the context of place-based watershed science problem solving. It includes five activities in the Innovative Technology in Science Inquiry (ITSI) portal (https://itsi.portal.concord.org/itsi#high-school-environmental-science). In the extensive teacher’s guide, teachers can utilize the instructional objectives and vocabulary list to showcase their student abilities and prior knowledge about their school’s context. For instance, teachers can include a discussion or activity to introduce their students to foundational understandings of watershed concepts.

Videos:

Publications:

Marcum-Dietrich, N., Kerlin, S., Staudt, C & Daniels, M. (2018). Our Watershed. The Science Teacher 85(2), 39-46.

Marcum-Dietrich, N. Hendrix, A, Kerlin, S., Daniels, M., & Staudt, C. (In review). Model My Watershed: An investigation into the role of big data, technology, and models in promoting student interest in watershed action.


Teaching Students to Reason about Variation and Covariation in Data: What do we Know, and What do we Need to Find Out?

(PI: Susan Kowalski)
Grades: Our synthesis and meta-analysis examines research across a wide range of grades, from elementary through college.
Target audience: Our target audience includes STEM education researchers with an interest in examining a wide variety of approaches to helping students develop an understanding of the concepts of variation and covariation.

Disciplines/Subject Areas: Mathematics, science, and engineering are the focal disciplines. Quantitative reasoning and analyzing and interpreting data are the practices of focus.

Project Description: An understanding of variation and covariation in data are foundational to understanding a wide variety of concepts in STEM. Mathematics, statistics, physics, Earth science, chemistry, engineering, and biology all require students to use and apply concepts of variation and covariation. Our primary goal is to better understand the current state of the art in helping students learn about variation and covariation. What approaches exist? What approaches have been tested empirically? What seems to be working well? Where are there gaps in our knowledge? These are questions we hope to answer with our synthesis and meta-analysis of STEM education research examining how students learn about variation and covariation.

Describe the data that students work with in your project: We are conducting a secondary synthesis and analysis of data collected from primary researchers in STEM education. Our data sources include published journal articles and research reports.

Key Challenge: We are uncovering a wide variety of studies that approach the concepts of variation and covariation in radically different ways. This is both a strength of the project and a challenge. We are currently grappling with how we can best summarize the disparate approaches to best inform the field of current research.


Related Resources

#NSTAChat on Data in the Science Classroom

View an archive of the #NSTAchat conversation held September 24, 2020. 

STEM for All Multiplex: Leveraging Authentic Data Across STEM Curricula

CADRE organized the Multiplex's Theme of the Month in September 2020 on Leveraging Authentic Data Across STEM Curricula, which included a blog, webinar panel, resources, and video playlist featuring DRK-12 and other federally funded work. Watch a recording of the webinar and/or view the online discussion.

Year