Quasi-experimental

Learning about Ecosystems Science and Complex Causality through Experimentation in a Virtual World

This project will develop a modified virtual world and accompanying curriculum for middle school students to help them learn to more deeply understand ecosystems patterns and the strengths and limitations of experimentation in ecosystems science. The project will build upon a computer world called EcoMUVE, a Multi-User Virtual Environment or MUVE, and will develop ways for students to conduct experiments within the virtual world and to see the results of those experiments.

Project Email: 
Lead Organization(s): 
Award Number: 
1416781
Funding Period: 
Mon, 09/01/2014 to Thu, 08/31/2017
Full Description: 

EcoXPT from videohall.com on Vimeo.

Comprehending how ecosystems function is important knowledge for citizens in making decisions and for students who aspire to become scientists. This understanding requires deep thinking about complex causality, unintended side-effects, and the strengths and limitations of experimental science. These are difficult concepts to learn due to the many interacting components and non-linear interrelationships involved. Ecosystems dynamics is particularly difficult to teach in classrooms because ecosystems involve complexities such as phenomena distributed widely across space that change over long time frames. Learning when and how experimental science can provide useful information in understanding ecosystems dynamics requires moving beyond the limited affordances of classrooms. The project will: 1) advance understanding of experimentation in ecosystems as it can be applied to education; 2) show how student learning is affected by having opportunities to experiment in the virtual world that simulate what scientists do in the real world and with models; and 3) produce results comparing this form of teaching to earlier instructional approaches. This project will result in a learning environment that will support learning about the complexities of the earth's ecosystem.

The project will build upon a computer world called EcoMUVE, a Multi-User Virtual Environment or MUVE, developed as part of an earlier NSF-funded project. A MUVE is a simulated world in which students can virtually walk around, make observations, talk to others, and collect data. EcoMUVE simulates a pond and a forest ecosystem. It offers an immersive context that makes it possible to teach about ecosystems in the classroom, allowing exploration of the complexities of large scale problems, extended time frames and and multiple causality. To more fully understand how ecosystems work, students need the opportunity to experiment and to observe what happens. This project will advance this earlier work by developing ways for students to conduct experiments within the virtual world and to see the results of those experiments. The project will work with ecosystem scientists to study the types of experiments that they conduct, informing knowledge in education about how ecosystem scientists think, and will build opportunities for students that mirror what scientists do. The project will develop a modified virtual world and accompanying curriculum for middle school students to help them learn to more deeply understand ecosystems patterns and the strengths and limitations of experimentation in ecosystems science. The resulting program will be tested against existing practice, the EcoMUVE program alone, and other programs that teach aspects of ecosystems dynamics to help teachers know how to best use these curricula in the classroom.

Teacher Professional Development for Technology-enhanced Inquiry to Foster Students' 21st Century Learning

This project will develop and evaluate a module for use in a 7th grade classroom that promotes student development of 21st Century skills with a particular focus on student development of scientific reasoning. The technology-enhanced curriculum will be designed to engage learners in deep and meaningful investigations to promote student learning of content in parallel with 21st century skills.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1417983
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

The goal of this Exploratory Design and Development Teaching project is to develop and evaluate a module for use in a 7th grade classroom that promotes student development of 21st Century skills with a particular focus on student development of scientific reasoning. The technology-enhanced curriculum will be designed to engage learners in deep and meaningful investigations to promote student learning of content in parallel with 21st century skills. The module will be designed using principles of inquiry-based learning as well as the principles of universal design for learning (UDL). The motivation behind this project is that it will directly contribute to the limited research on the interventions that impact teachers' capacity to provide high quality 21st century STEM education to all students, with a specific focus on underrepresented minorities and those with disabilities. The classroom setting for which the curriculum will be delivered is within an urban district which includes a large number of minority students and over 20% students with specific learning disabilities. The project will catalyze students' deep understanding of content knowledge while developing 21st century skills in parallel; hence better preparing students for sustainable learning experiences into high school and beyond.

A study will be conducted to determine the effectiveness of the learning modules on classroom practices as well as student learning. A mixed methods design involving multiple measures will provide insights into changes in teachers' content knowledge, teaching practices that include a focus on 21st century learning, and fidelity of use of the TI21 framework for implementation of the learning activities. Pre- and post-testing of students using a scientific reasoning assessment and surveys on attitudes towards STEM, along with validated and widely used concept inventories, will provide further measures. As part of this exploratory project, the design and validity of instruments for use with the targeted population, which includes students with specific learning disabilities, will be further tested. This will include administering some of the assessments through web-based apps to meet the needs of these students. The learning modules, with embedded assessments and web-based apps, will provide an innovative approach in which transferable 21st century skills can be developed and measured. Outcomes of this project will be disseminated throughout the urban school system and therefore have the ability to impact thousands of other students (mostly minorities and many with disabilities) and their science, math, and technology teachers. Project outcomes will also inform the development of future science and/or modules for use in similar urban classroom settings.

Reclaiming Access to Inquiry-based Science Education (RAISE) for Incarcerated Students

This project will develop a Universal Design for Learning, project-based inquiry science program that includes virtual learning environments, virtual laboratories, and digital scaffolds and supports that promote scientific learning for incarcerated youth.

Award Number: 
1418152
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

This project is unique in targeting arguably the most vulnerable learners in the American education system: youth confined in juvenile corrections facilities. Three primary problems confronting science education in these settings are: (1) inadequate curriculum and resources; (2) inadequately prepared and supported teachers; and (3) a heterogeneous group of learners, many of whom have disabilities, are disengaged, and/or lack reading and mathematics skills. Failure to address these challenges and the broader educational needs of incarcerated juveniles has broad implications for society, so this project is timely and has high potential for broad impacts.

To address these problems project personnel will employ an iterative development process to develop a curriculum designed to increase access to and mastery of science content, concepts, and inquiry skills critical for careers in the 21st Century STEM workforce. They will then prepare teachers to implement the program in pilot testing in juvenile corrections facilities in Massachusetts. Specifically, the investigators will: (1) align and adapt an existing biology curriculum using Common Core State Standards and Universal Design for Learning principles; (2) develop all materials, digital supports and scaffolds, virtual learning environments and labs, assessments, and teacher professional development materials for one curriculum unit; (3) conduct usability evaluation of all materials and use the results to refine and finalize two curriculum units; (4) prepare teachers to implement the biology program in juvenile corrections education settings; (5) conduct a quasi-experimental study to examine the impacts of the biology program on the content knowledge and inquiry skills of students, their interests, and their levels of engagement; and, (6) disseminate the findings to various constituency groups. The final product will be a Universal Design for Learning, project-based inquiry science program that includes virtual learning environments, virtual laboratories, and digital scaffolds and supports that promote scientific learning for incarcerated youth.

Preparing Urban Middle Grades Mathematics Teachers to Teach Argumentation Throughout the School Year

The objective of this project is to develop a toolkit of resources and practices that will help inservice middle grades mathematics teachers support mathematical argumentation throughout the school year. A coherent, portable, two-year-long professional development program on mathematical argumentation has the potential to increase access to mathematical argumentation for students nationwide and, in particular, to address the needs of teachers and students in urban areas.

Lead Organization(s): 
Award Number: 
1417895
Funding Period: 
Sun, 06/15/2014 to Thu, 05/31/2018
Full Description: 

The project is an important study that builds on prior research to bring a comprehensive professional development program to another urban school district, The District of Columbia Public Schools. The objective of this full research and development project is to develop a toolkit  that provides resources and practices for inservice middle grades mathematics teachers to support mathematical argumentation throughout the school year. Mathematical argumentation, the construction and critique of mathematical conjectures and justifications, is a fundamental disciplinary practice in mathematics that students often never master. Building on a proof of concept of the project's approach ifrom two prior NSF-funded studies, this project expands the model to help teachers support mathematical argumentation all year. A coherent, portable, two-year-long professional development program on mathematical argumentation has the potential to increase access to mathematical argumentation for students nationwide and, in particular, to address the needs of teachers and students in urban areas. Demonstrating this program in the nation's capital will likely attract broad interest and produces important knowledge about how to implement mathematical practices in urban settings. Increasing mathematical argumentation in schools has the potential for dramatic contributions to students' achievement and participation in 21st century workplaces.

Mathematical argumentation is rich discussion in which students take on mathematical authority and co-construct conjectures and justifications. For many teachers, supporting such discourse is challenging; many are most comfortable with Initiate-Respond-Evaluate types of practices and/or have insufficient content understanding. The professional development trains teachers to be disciplined improvisers -- professionals with a toolkit of tools, knowledge, and practices to be deployed creatively and responsively as mathematical argumentation unfolds. This discipline includes establishing classroom norms and planning lessons for argumentation. The model's theory of action has four design principles: provide the toolkit, use simulations of the classroom to practice improvising, support learning of key content, and provide job-embedded, technology-enabled supports for using new practices all year. Three yearlong studies will address design, feasibility, and promise. In Study 1 the team co-designs tools with District of Columbia Public Schools staff. Study 2 is a feasibility study to examine program implementation, identify barriers and facilitators, and inform improvements. Study 3 is a quasi-experimental pilot to test the promise for achieving intended outcomes: expanding teachers' content knowledge and support of mathematical argumentation, and increasing students' mathematical argumentation in the classroom and spoken argumentation proficiency. The studies will result in a yearlong professional development program with documentation of the theory of action, design decisions, pilot data, and instrument technical qualities.

Investigating How to Enhance Scientific Argumentation through Automated Feedback in the Context of Two High School Earth Science Curriculum Units

This project responds to the need for technology-enhanced assessments that promote the critical practice of scientific argumentation--making and explaining a claim from evidence about a scientific question and critically evaluating sources of uncertainty in the claim. It will investigate how to enhance this practice through automated scoring and immediate feedback in the context of two high school curriculum units--climate change and fresh-water availability--in schools with diverse student populations. 

Lead Organization(s): 
Award Number: 
1418019
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

With the current emphasis on learning science by actively engaging in the practices of science, and the call for integration of instruction and assessment; new resources, models, and technologies are being developed to improve K-12 science learning. Student assessment has become a nationwide educational priority due, in part, to the need for relevant and timely data that inform teachers, administrators, researchers, and the public about how all students perform and think while learning science. This project responds to the need for technology-enhanced assessments that promote the critical practice of scientific argumentation--making and explaining a claim from evidence about a scientific question and critically evaluating sources of uncertainty in the claim. It will investigate how to enhance this practice through automated scoring and immediate feedback in the context of two high school curriculum units--climate change and fresh-water availability--in schools with diverse student populations. The project will apply advanced automated scoring tools to students' written scientific arguments, provide individual students with customized feedback, and teachers with class-level information to assist them with improving scientific argumentation. The key outcome of this effort will be a technology-supported assessment model of how to advance the understanding of argumentation, and the use of multi-level feedback as a component of effective teaching and learning. The project will strengthen the program's current set of funded activities on assessment, focusing these efforts on students' argumentation as a complex science practice.

This design and development research targets high school students (n=1,940) and teachers (n=22) in up to 10 states over four years. The research questions are: (1) To what extent can automated scoring tools, such as c-rater and c-rater-ML, diagnose students' explanations and uncertainty articulations as compared to human diagnosis?; (2) How should feedback be designed and delivered to help students improve scientific argumentation?; (3) How do teachers use and interact with class-level automated scores and feedback to support students' scientific argumentation with real-data and models?; and (4) How do students perceive their overall experience with the automated scores and immediate feedback when learning core ideas in climate change and fresh-water availability topics through scientific argumentation enhanced with modeling? In Years 1 and 2, plans are to conduct feasibility studies to build automated scoring models and design feedback for previously tested assessments for the two curriculum units. In Year 3, the project will implement design studies in order to identify effective feedback through random assignment. In Year 4, a pilot study will investigate if effective feedback should be offered with or without scores. The project will employ a mixed-methods approach. Data-gathering strategies will include classroom observations; screencast and log data of teachers' and students' interaction with automated feedback; teachers' and students' surveys with selected- and open-ended questions; and in-depth interviews with teachers and students. All constructed-response explanations and uncertainty items will be scored using automated scoring engines with fine-grained rubrics. Data analysis strategies will include multiple criteria to evaluate the quality of automated scores; descriptive statistical abalyses; analysis of variance to investigate differences in outcomes from the designed studies' pre/posttests and embedded assessments; analysis of covariance to investigate student learning trajectories; two-level hierarchical linear modeling to study the clustering of students within a class; and analysis of screencasts and log data.

GRIDS: Graphing Research on Inquiry with Data in Science

The Graphing Research on Inquiry with Data in Science (GRIDS) project will investigate strategies to improve middle school students' science learning by focusing on student ability to interpret and use graphs. GRIDS will undertake a comprehensive program to address the need for improved graph comprehension. The project will create, study, and disseminate technology-based assessments, technologies that aid graph interpretation, instructional designs, professional development, and learning materials.

Award Number: 
1418423
Funding Period: 
Mon, 09/01/2014 to Sat, 08/31/2019
Full Description: 

The Graphing Research on Inquiry with Data in Science (GRIDS) project is a four-year full design and development proposal, addressing the learning strand, submitted to the DR K-12 program at the NSF. GRIDS will investigate strategies to improve middle school students' science learning by focusing on student ability to interpret and use graphs. In middle school math, students typically graph only linear functions and rarely encounter features used in science, such as units, scientific notation, non-integer values, noise, cycles, and exponentials. Science teachers rarely teach about the graph features needed in science, so students are left to learn science without recourse to what is inarguably a key tool in learning and doing science. GRIDS will undertake a comprehensive program to address the need for improved graph comprehension. The project will create, study, and disseminate technology-based assessments, technologies that aid graph interpretation, instructional designs, professional development, and learning materials.

GRIDS will start by developing the GRIDS Graphing Inventory (GGI), an online, research-based measure of graphing skills that are relevant to middle school science. The project will address gaps revealed by the GGI by designing instructional activities that feature powerful digital technologies including automated guidance based on analysis of student generated graphs and student writing about graphs. These materials will be tested in classroom comparison studies using the GGI to assess both annual and longitudinal progress. Approximately 30 teachers selected from 10 public middle schools will participate in the project, along with approximately 4,000 students in their classrooms. A series of design studies will be conducted to create and test ten units of study and associated assessments, and a minimum of 30 comparison studies will be conducted to optimize instructional strategies. The comparison studies will include a minimum of 5 experiments per term, each with 6 teachers and their 600-800 students. The project will develop supports for teachers to guide students to use graphs and science knowledge to deepen understanding, and to develop agency and identity as science learners.

From Elementary Generalist to Mathematics Specialist: Examining Teacher Practice and Student Outcomes in Departmental and Self-Contained Models

This research investigates student mathematics learning outcomes at the elementary level in relation to teacher expertise (elementary teachers with math specialist certification versus generally prepared elementary teachers) and school organization (departmentalized versus self-contained mathematics classrooms). Findings will provide evidence of the impact of content-specific teacher expertise and a departmentalized school organizational model that offers students access to well-qualified teachers of mathematics with no additional staffing costs.

Lead Organization(s): 
Award Number: 
1414438
Funding Period: 
Fri, 08/01/2014 to Tue, 07/31/2018
Full Description: 

This research investigates student mathematics learning outcomes at the elementary level in relation to teacher expertise (elementary teachers with math specialist certification versus generally prepared elementary teachers) and school organization (departmentalized versus self-contained mathematics classrooms). University of Missouri researchers will organize and facilitate the research in multiple Missouri public and private school sites. Findings will provide evidence of the impact of content-specific teacher expertise and a departmentalized school organizational model that offers students access to well-qualified teachers of mathematics with no additional staffing costs. To investigate the impact of teacher expertise and school organization on student learning experimental, quasi-experimental, and qualitative designs are employed. Specifically, 80 teachers who have earned state certification as Elementary Mathematics Specialists (hereafter, "EMS teachers") will be selected to participate in the study. Employing a randomized experimental design, half of the EMS teachers will be assigned to teach in a departmental model (Condition 1) in their school, teaching two or more sections of grade 4 mathematics. The other half will remain in generalist (hereafter, self-contained) positions (Condition 2) in their school, teaching all regular subjects to a single class of students. A comparison group of 40 non-EMS teachers with self-contained teaching assignments (Condition 3) will be selected from the same schools in Condition 2. The Smarter Balanced assessment will provide a baseline measure of students' prior achievement in grade 3 and also a measure of the mathematics achievement of grade 4 students taught by the 120 teachers in the study. The project team will analyze student-level mathematics scores linked with specific teachers.

Improving student achievement in mathematics at the elementary level is particularly challenging due to the way elementary students are generally organized for instruction (e.g., one teacher responsible for teaching all subjects to 25-30 students). Because elementary teachers must be knowledgeable about many content areas, they rarely study mathematics in depth, even though there is a growing body of evidence showing the impact of specialized mathematical knowledge for teaching on student learning outcomes. This study carefully researches an alternative model, where elementary teachers with special training in mathematics teaching and learning are assigned more than one single class of students for mathematics instruction. Findings from the research will provide evidence about the impact of both certification as an elementary mathematics specialist and teaching in a departmental assignment on student learning. The results will help school and district leaders better plan for assignment of staff to provide all students with access to high quality mathematics instruction.

EarSketch: An Authentic, Studio-based STEAM Approach to High School Computing Education

This project will study the influence on positive student achievement and engagement (particularly among populations traditionally under-represented in computer science) of an intervention that integrates a computational music remixing tool -EarSketch- with the Computer Science Principles, a view of computing literacy that is emerging as a new standard for Advanced Placement and other high school computer science courses.

Award Number: 
1417835
Funding Period: 
Fri, 08/01/2014 to Tue, 07/31/2018
Project Evaluator: 
Mary Moriarity
Full Description: 

This project will study the influence on positive student achievement and engagement (particularly among populations traditionally under-represented in computer science) of an intervention that integrates a computational music remixing tool -EarSketch- with the Computer Science Principles, a view of computing literacy that is emerging as a new standard for Advanced Placement and other high school computer science courses. The project is grounded on the premise that EarSketch, a STEM + Art (STEAM) learning environment, embodies authenticity (i.e., its cultural and industry relevance in both arts and STEM domains), along with a context that facilitates communication and collaboration among students (i.e., through a studio-based learning approach). These elements are critical to achieving successful outcomes across diverse student populations. Using agent-based modeling, the research team will investigate what factors enhance or impede implementation of authentic STEAM tools in different school settings.

The researchers will be engaged in a multi-stage process to develop: a) an implementation-ready, web-based EarSketch learning environment that integrates programming, digital audio workstation, curriculum, audio loop library, and social sharing features, along with studio-based learning functionality to support student presentation, critique, discussion, and collaboration; and b) an online professional learning course for teachers adopting EarSketch in Computer Science Principles courses. Using these resources, the team will conduct a quasi-experimental study of EarSketch in Computer Science Principles high school courses across the state of Georgia; measure student learning and engagement across multiple demographic categories; and determine to what extent an EarSketch-based CS Principles course promotes student achievement and engagement across different student populations. The project will include measures of student performance, creativity, collaboration, and communication in student programming tasks to determine the extent to which studio-based learning in EarSketch promotes success in these important areas. An agent-based modeling framework in multiple school settings will be developed to determine what factors enhance or impede implementation of EarSketch under conditions of routine practice.

Developing and Testing the Internship-inator, a Virtual Internship in STEM Authorware System

The Internship-inator is an authorware system for developing and testing virtual internships in multiple STEM disciplines. In a virtual internship, students are presented with a complex, real-world STEM problem for which there is no optimal solution. Students work in project teams to read and analyze research reports, design and perform experiments using virtual tools, respond to the requirements of stakeholders and clients, write reports and present and justify their proposed solutions. 

Award Number: 
1418288
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

Ensuring that students have the opportunities to experience STEM as it is conducted by scientists, mathematicians and engineers is a complex task within the current school context. This project will expand access for middle and high school students to virtual internships, by enabling STEM content developers to design and customize virtual internships. The Internship-inator is an authorware system for developing and testing virtual internships in multiple STEM disciplines. In a virtual internship, students are presented with a complex, real-world STEM problem for which there is no optimal solution. Students work in project teams to read and analyze research reports, design and perform experiments using virtual tools, respond to the requirements of stakeholders and clients, write reports and present and justify their proposed solutions. The researchers in this project will work with a core development network to develop and refine the authorware, constructing up to a hundred new virtual internships and a user group of more than 70 STEM content developers. The researchers will iteratively analyze the performance of the authorware, focusing on optimizing the utility and the feasibility of the system to support virtual internship development. They will also examine the ways in which the virtual internships are implemented in the classroom to determine the quality of the STEM internship design and influence on student learning.

The Intership-inator builds on over ten years of NSF support for the development of Syntern, a platform for deploying virtual internships that has been used in middle schools, high schools, informal science programs, and undergraduate education. In the current project, the researchers will recruit two waves of STEM content developers to expand their current core development network. A design research perspective will be used to examine the ways in which the developers interact with the components of the authorware and to document the influence of the virtual internships on student learning. The researchers will use a quantitative ethnographic approach to integrate qualitative data from surveys and interviews with the developers with their quantitative interactions with the authorware and with student use and products from pilot and field tests of the virtual internships. Data-mining and learning analytics will be used in combination with hierarchical linear modeling, regression techniques and propensity score matching to structure the quasi-experimental research design. The authorware and the multiple virtual internships will provide researchers, developers, and teachers a rich learning environment in which to explore and support students' learning of important college and career readiness content and disciplinary practices. The findings of the use of the authorware will inform STEM education about the important design characteristics for authorware that supports the work of STEM content and curriculum developers.

Climate Change Narrative Game Education (CHANGE)

This exploratory project helps high school students learn complex Global Climate Change (GCC) science by making it personally relevant and understandable. CHANGE creates a prototype curriculum, and integrates it into elective Marine Sciences high school courses. Research will examine the project's impact on student learning of climate science, student attitude toward science, and teacher instruction of climate science.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1316782
Funding Period: 
Sun, 09/15/2013 to Wed, 08/31/2016
Full Description: 

This exploratory project helps high school students learn complex Global Climate Change (GCC) science by making it personally relevant and understandable. CHANGE creates a prototype curriculum, and integrates it into elective Marine Sciences high school courses. Research will examine the project's impact on student learning of climate science, student attitude toward science, and teacher instruction of climate science. The goal of this project is to develop a place-based futuristic gaming simulation model that can easily extend to the other locales in other states, based on local climate change effects, local stakeholders, local economic and social effects to motivate the high school students in that area. CHANGE uses: (a) scientifically realistic text narratives about future Florida residents (text stories with local Florida characters, many years in the future based on GCC), (b) local, place-based approach grounded in west-central Florida Gulf Coast using scientific data, (c) a focus on the built environment, (d) simulations & games based on scientific data to help students learn principles of GCC so students can experience and try to cope with the potential long term effect of GCC via role-play and science-based simulation, and (e) a web-based eBook narrative where sections of narrative text alternate with simulations/computer games. The proposed project will work with 25 high school Marine Science teachers in 25 schools in Hillsborough County, Florida. The project delivers new research for instructional technologists and serious game developers regarding effective interface and usability design of intermedia narrative gaming-simulations for education.

This project employs and researches innovative models for delivering high school GCC education. GCC is a complex topic involving numerous factors and uncertainties making teaching this extremely important topic very difficult. The pioneering techniques proposed for this project will advance science education of GCC. It also will deliver new research for instructional technologists and serious game developers regarding effective interface and usability design of intermedia narrative gaming-simulations for education. Effective education is probably the most crucial part in our ability to cope with climate change. CHANGE will educate underserved low SES and minority high school students in Hillsborough County, and later elsewhere, with a model making GCC personally relevant to them.

Pages

Subscribe to Quasi-experimental