Assessment

Taking Games to School: Exploratory Study to Support Game-based Teaching and Learning In High-School Science Classes

This project is building a set of software tools, including a tool for annotating screen recordings of activities in games, a teacher data dashboard for information about students' in-game learning, and tools to help teachers customize activities in games to better align with curricular standards. The project will find out whether these new tools can enhance teaching and/or learning. 

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1415284
Funding Period: 
Tue, 07/15/2014 to Sat, 06/30/2018
Full Description: 

Research shows that educational games can enhance students' science learning, but current work leaves teachers dependent on researchers and games companies to provide good games and game-based curricula. This project aims to study how teachers can be involved in making science learning games more effective, and how educational science games can better support good teaching. This project is building a set of software tools, including a tool for annotating screen recordings of activities in games, a teacher data dashboard for information about students' in-game learning, and tools to help teachers customize activities in games to better align with curricular standards. It will conduct studies with successful research-based educational games for learning science, and popularly available educational games from websites such as BrainPop, in a network of teachers who have experience using 'canned' games in their classrooms. The project will find out whether these new tools can enhance teaching and/or learning. It will also help develop a list of the types of customization options teachers need in order to be able to effectively use educational games in their classrooms. If successful, this research could point the way towards new tools that let teachers create activities that turn any game into an educational game, and to better use existing educational games in their classrooms. This could greatly speed up our ability to deliver high-quality learning experiences through educational games.

This project involves a participatory design process in which a small number of experienced teachers will feed into a principled, iterative refinement of prototypes of the tools (annotation, data dashboard, and level-builder) to be prototyped within the Brainplay suite. In the beta testing phase, a hierarchical linear model analysis will be conducted on both student and teacher outcomes in 25 classrooms. In addition to the quantitative analysis, qualitative studies involving classroom observations, focus groups, and teacher journaling will be conducted to examine impact on teaching practices and refine the functional specifications. Project dissemination will take place through the community around the previously-developed Leveling Up games (played around 10,000 times per week), and through existing professional networks such as Edmodo. The project will also work within the games community to help inform possible approaches to logging learning data and allowing teacher customization across all games.

Reclaiming Access to Inquiry-based Science Education (RAISE) for Incarcerated Students

This project will develop a Universal Design for Learning, project-based inquiry science program that includes virtual learning environments, virtual laboratories, and digital scaffolds and supports that promote scientific learning for incarcerated youth.

Award Number: 
1418152
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

This project is unique in targeting arguably the most vulnerable learners in the American education system: youth confined in juvenile corrections facilities. Three primary problems confronting science education in these settings are: (1) inadequate curriculum and resources; (2) inadequately prepared and supported teachers; and (3) a heterogeneous group of learners, many of whom have disabilities, are disengaged, and/or lack reading and mathematics skills. Failure to address these challenges and the broader educational needs of incarcerated juveniles has broad implications for society, so this project is timely and has high potential for broad impacts.

To address these problems project personnel will employ an iterative development process to develop a curriculum designed to increase access to and mastery of science content, concepts, and inquiry skills critical for careers in the 21st Century STEM workforce. They will then prepare teachers to implement the program in pilot testing in juvenile corrections facilities in Massachusetts. Specifically, the investigators will: (1) align and adapt an existing biology curriculum using Common Core State Standards and Universal Design for Learning principles; (2) develop all materials, digital supports and scaffolds, virtual learning environments and labs, assessments, and teacher professional development materials for one curriculum unit; (3) conduct usability evaluation of all materials and use the results to refine and finalize two curriculum units; (4) prepare teachers to implement the biology program in juvenile corrections education settings; (5) conduct a quasi-experimental study to examine the impacts of the biology program on the content knowledge and inquiry skills of students, their interests, and their levels of engagement; and, (6) disseminate the findings to various constituency groups. The final product will be a Universal Design for Learning, project-based inquiry science program that includes virtual learning environments, virtual laboratories, and digital scaffolds and supports that promote scientific learning for incarcerated youth.

Learning Linkages: Integrating Data Streams of Multiple Modalities and Timescales (Collaborative Research: Sherin)

In this project, researchers will collaborate to enhance understanding of influences on learning, and improve teaching and learning in high school and middle school STEM classes. They will leverage the latest tools for data processing and many different streams of data that can be collected in technology-rich classrooms to (1) identify classroom factors that affect learning and (2) explore how to use that data to automatically track development of students' understanding and capabilities over time.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1418020
Funding Period: 
Mon, 09/01/2014 to Thu, 08/31/2017
Full Description: 

This Research on Education and Learning (REAL) project arises from an October 2014 Ideas Lab on Data-intensive Research to Improve Teaching and Learning. The intentions of that effort were to (1) bring together researchers from across disciplines to foster novel, transformative, multidisciplinary approaches to using the data in large education-related data sets to create actionable knowledge for improving STEM teaching and learning environments in the medium term; and (2) revolutionize learning in the longer term. In this project, researchers from Carnegie-Mellon University, Wested, Arizona State University, and Northwestern University will collaborate to enhance understanding of influences on learning, and improve teaching and learning in high school and middle school STEM classes. To accomplish this, they will leverage the latest tools for data processing and many different streams of data that can be collected in technology-rich classrooms to (1) identify classroom factors that affect learning and (2) explore how to use that data to automatically track development of students' understanding and capabilities over time.

Two forces are poised to transform research on learning. First, more and more student work is conducted on computers and online, producing vast amounts of learning-related data. At the same time, advances in computing, data mining, and learning analytics are providing new tools for the collection, analysis, and representation of these data. Together, the available data and analytical tools enable smart and responsive systems that personalize learning experiences for individual learners. The PIs aim to collect highly enriched data that go far beyond typical computer data capture, leveraging the latest tools for data processing to generate new insights about STEM teaching and learning. Working to maximize the potential while mitigating the risks of automated data collection and analysis, they will: (1) collect and integrate diverse sources of data including log files, videos, and written artifacts from across eight different two-week enactments of two different computer supported learning environments (one used in middle school math and one in high school science); and (2) compare analyses of log-file data with analyses of integrated datasets to understand the possibilities and limitations in using log-file data for assessment of student learning and proficiency. The collaborators expect their findings will inform both theories and practical recommendations applicable across a wide range of disciplines and settings.

Investigating How to Enhance Scientific Argumentation through Automated Feedback in the Context of Two High School Earth Science Curriculum Units

This project responds to the need for technology-enhanced assessments that promote the critical practice of scientific argumentation--making and explaining a claim from evidence about a scientific question and critically evaluating sources of uncertainty in the claim. It will investigate how to enhance this practice through automated scoring and immediate feedback in the context of two high school curriculum units--climate change and fresh-water availability--in schools with diverse student populations. 

Lead Organization(s): 
Award Number: 
1418019
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

With the current emphasis on learning science by actively engaging in the practices of science, and the call for integration of instruction and assessment; new resources, models, and technologies are being developed to improve K-12 science learning. Student assessment has become a nationwide educational priority due, in part, to the need for relevant and timely data that inform teachers, administrators, researchers, and the public about how all students perform and think while learning science. This project responds to the need for technology-enhanced assessments that promote the critical practice of scientific argumentation--making and explaining a claim from evidence about a scientific question and critically evaluating sources of uncertainty in the claim. It will investigate how to enhance this practice through automated scoring and immediate feedback in the context of two high school curriculum units--climate change and fresh-water availability--in schools with diverse student populations. The project will apply advanced automated scoring tools to students' written scientific arguments, provide individual students with customized feedback, and teachers with class-level information to assist them with improving scientific argumentation. The key outcome of this effort will be a technology-supported assessment model of how to advance the understanding of argumentation, and the use of multi-level feedback as a component of effective teaching and learning. The project will strengthen the program's current set of funded activities on assessment, focusing these efforts on students' argumentation as a complex science practice.

This design and development research targets high school students (n=1,940) and teachers (n=22) in up to 10 states over four years. The research questions are: (1) To what extent can automated scoring tools, such as c-rater and c-rater-ML, diagnose students' explanations and uncertainty articulations as compared to human diagnosis?; (2) How should feedback be designed and delivered to help students improve scientific argumentation?; (3) How do teachers use and interact with class-level automated scores and feedback to support students' scientific argumentation with real-data and models?; and (4) How do students perceive their overall experience with the automated scores and immediate feedback when learning core ideas in climate change and fresh-water availability topics through scientific argumentation enhanced with modeling? In Years 1 and 2, plans are to conduct feasibility studies to build automated scoring models and design feedback for previously tested assessments for the two curriculum units. In Year 3, the project will implement design studies in order to identify effective feedback through random assignment. In Year 4, a pilot study will investigate if effective feedback should be offered with or without scores. The project will employ a mixed-methods approach. Data-gathering strategies will include classroom observations; screencast and log data of teachers' and students' interaction with automated feedback; teachers' and students' surveys with selected- and open-ended questions; and in-depth interviews with teachers and students. All constructed-response explanations and uncertainty items will be scored using automated scoring engines with fine-grained rubrics. Data analysis strategies will include multiple criteria to evaluate the quality of automated scores; descriptive statistical abalyses; analysis of variance to investigate differences in outcomes from the designed studies' pre/posttests and embedded assessments; analysis of covariance to investigate student learning trajectories; two-level hierarchical linear modeling to study the clustering of students within a class; and analysis of screencasts and log data.

Improving Students' Mathematical Proficiency through Formative Assessment: Responding to an Urgent Need in the Common Core Era

The overarching goal of this RAPID project is to contribute to the national goal of improving students' mathematical proficiency by providing information and guidance to mathematics education practitioners and scholars to support a sharpened focus on formative assessment. The project produces, analyzes, and makes available to the field timely information regarding the views and practices of mathematics teacher educators and professional development specialists regarding formative assessment early in the enactment of ambitious standards in mathematics.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1439366
Funding Period: 
Sun, 06/15/2014 to Sun, 05/31/2015
Full Description: 

The products of this project will be useful to national organizations, their state and local affiliates, and school districts as they plan and offer mathematics professional development to support the implementation of high quality mathematics instruction to meet the urgent national need for smart and effective approaches to support ambitious college and career-ready standards in mathematics. Directing mathematics instruction toward ambitious learning goals is intended to address the critically important national priority of improving students' mathematics achievement. It is widely recognized that successful attainment of the content and practices contained in any ambitious set of learning goals, requires well-designed, smartly delivered, professional development for the nation's mathematics teachers. The information generated from this project is critical to inform nationwide mathematics professional development to support the implementation of ambitious mathematics learning goals. For our nation's teachers and students to attain ambitious learning goals, it is imperative that formative assessment becomes a more prominent feature of mathematics instruction as there is an evidence base that suggests formative assessment positively impacts student learning.

The overarching goal of this RAPID project is to contribute to the national goal of improving students' mathematical proficiency by providing much-needed information and guidance to mathematics education practitioners and scholars to support a sharpened focus on formative assessment. The project produces, analyzes, and makes available to the field timely information regarding the views and practices of mathematics teacher educators and professional development specialists regarding formative assessment early in the enactment of ambitious standards in mathematics. Moreover, it offers a potentially transformative view of formative assessment as integrated with other promising mathematics instructional frameworks, approaches and practices that have already established a strong presence in the mathematics education community and have influenced the instructional practice of many teacher educators and teachers. The project will result in: (a) an in-depth analysis of the responses of mathematics teacher educators and professional development specialists to a recent survey that probed their practices and beliefs related to formative assessment and its intertwined relationships with promising mathematics instructional frameworks, approaches and practices; (b) collaborative work among mathematics teacher educators and professional development specialists to elaborate effective ways to focus on formative assessment in the preparation and continuing education of teachers of mathematics; and (c) a set of design features and principles, along with associated activities, intended to undergird creating and sustaining an approach to mathematics teacher professional development that both attends to critically important instructional practices of formative assessment and links to other promising mathematics instructional frameworks, approaches and practices.

GRIDS: Graphing Research on Inquiry with Data in Science

The Graphing Research on Inquiry with Data in Science (GRIDS) project will investigate strategies to improve middle school students' science learning by focusing on student ability to interpret and use graphs. GRIDS will undertake a comprehensive program to address the need for improved graph comprehension. The project will create, study, and disseminate technology-based assessments, technologies that aid graph interpretation, instructional designs, professional development, and learning materials.

Award Number: 
1418423
Funding Period: 
Mon, 09/01/2014 to Sat, 08/31/2019
Full Description: 

The Graphing Research on Inquiry with Data in Science (GRIDS) project is a four-year full design and development proposal, addressing the learning strand, submitted to the DR K-12 program at the NSF. GRIDS will investigate strategies to improve middle school students' science learning by focusing on student ability to interpret and use graphs. In middle school math, students typically graph only linear functions and rarely encounter features used in science, such as units, scientific notation, non-integer values, noise, cycles, and exponentials. Science teachers rarely teach about the graph features needed in science, so students are left to learn science without recourse to what is inarguably a key tool in learning and doing science. GRIDS will undertake a comprehensive program to address the need for improved graph comprehension. The project will create, study, and disseminate technology-based assessments, technologies that aid graph interpretation, instructional designs, professional development, and learning materials.

GRIDS will start by developing the GRIDS Graphing Inventory (GGI), an online, research-based measure of graphing skills that are relevant to middle school science. The project will address gaps revealed by the GGI by designing instructional activities that feature powerful digital technologies including automated guidance based on analysis of student generated graphs and student writing about graphs. These materials will be tested in classroom comparison studies using the GGI to assess both annual and longitudinal progress. Approximately 30 teachers selected from 10 public middle schools will participate in the project, along with approximately 4,000 students in their classrooms. A series of design studies will be conducted to create and test ten units of study and associated assessments, and a minimum of 30 comparison studies will be conducted to optimize instructional strategies. The comparison studies will include a minimum of 5 experiments per term, each with 6 teachers and their 600-800 students. The project will develop supports for teachers to guide students to use graphs and science knowledge to deepen understanding, and to develop agency and identity as science learners.

Focus on Energy: Preparing Elementary Teachers to Meet the NGSS Challenge (Collaborative Research: Vokos)

This project will develop and investigate the opportunities and limitations of Focus on Energy, a professional development (PD) system for elementary teachers (grades 3-5). The PD will contain: resources that will help teachers to interpret, evaluate and cultivate students' ideas about energy; classroom activities to help them to identify, track and represent energy forms and flows; and supports to help them in engaging students in these activities.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1418211
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

The Next Generation Science Standards (NGSS) identify an ambitious progression for learning energy, beginning in elementary school. To help the nation's teachers address this challenge, this project will develop and investigate the opportunities and limitations of Focus on Energy, a professional development (PD) system for elementary teachers (grades 3-5). The PD will contain: resources that will help teachers to interpret, evaluate and cultivate students' ideas about energy; classroom activities to help them to identify, track and represent energy forms and flows; and supports to help them in engaging students in these activities. Teachers will receive the science and pedagogical content knowledge they need to teach about energy in a crosscutting way across all their science curricula; students will be intellectually engaged in the practice of developing, testing, and revising a model of energy they can use to describe phenomena both in school and in their everyday lives; and formative assessment will guide the moment-by-moment advancement of students' ideas about energy.

This project will develop and test a scalable model of PD that will enhance the ability of in-service early elementary teachers to help students learn energy concepts by coordinating formative assessment, face-to-face and web-based PD activities. Researchers will develop and iteratively refine tools to assess both teacher and student energy reasoning strategies. The goals of the project include (1) teachers' increased facility with, and disciplined application of, representations and energy reasoning to make sense of everyday phenomena in terms of energy; (2) teachers' increased ability to interpret student representations and ideas about energy to make instructional decisions; and (3) students' improved use of representations and energy reasoning to develop and refine models that describe energy forms and flows associated with everyday phenomena. The web-based product will contain: a set of formative assessments to help teachers to interpret student ideas about energy based on the Facets model; a series of classroom tested activities to introduce the Energy Tracking Lens (method to explore energy concept using multiple representations); and videos of classroom exemplars as well as scientists thinking out loud while using the Energy Tracking Lens. The project will refine the existing PD and build a system that supports online implementation by constructing a facilitator's guide so that the online community can run with one facilitator.

Empowering Teachers through VideoReview

This project  will develop a video recording and analysis system called VideoReView (VRV) that allows grade four science teachers to record, tag, and analyze video in their classroom in real time. The investigators will then study and enhance the system in the context of professional learning communities of teachers. 

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1415898
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

This project represents a collaboration between TERC and IntuVision to develop a video recording and analysis system called VideoReView (VRV) that allows grade four science teachers to record, tag, and analyze video in their classroom in real time. The system will contain a number of features---such as a sophisticated system of tagging and the automatic detection of important video segments---designed to speed and assist the teacher in its use. The investigators will then study and enhance the system in the context of professional learning communities of teachers. The system is expected to enable teachers to examine their own teaching, and that of others, in a much more dynamic and specific way and to integrate video into their ongoing structures of professional learning. To date, video analysis of teaching is out of the reach of ordinary teachers. If successful, this research could change the way teachers engage in their own profession and their understanding of, for example, student thinking and argumentation in science---something emphasized in the Next Generation Science Standards---but previously more difficult to do without being able to replay and refine teaching episodes.

The complete VRV System will be tested with 18 Grade four teachers and approximately 400 students from six schools in the Newton Public School System in a Boston suburb. The emphasis of the study will be on the ability of teachers to use the system with little outside assistance, means of enhancing its features and usability, and its integration into professional learning communities. A mixed methods research design will be used that includes surveys and interviews. The study outcomes will be disseminated through publications and conference presentations.

EarSketch: An Authentic, Studio-based STEAM Approach to High School Computing Education

This project will study the influence on positive student achievement and engagement (particularly among populations traditionally under-represented in computer science) of an intervention that integrates a computational music remixing tool -EarSketch- with the Computer Science Principles, a view of computing literacy that is emerging as a new standard for Advanced Placement and other high school computer science courses.

Award Number: 
1417835
Funding Period: 
Fri, 08/01/2014 to Tue, 07/31/2018
Project Evaluator: 
Mary Moriarity
Full Description: 

This project will study the influence on positive student achievement and engagement (particularly among populations traditionally under-represented in computer science) of an intervention that integrates a computational music remixing tool -EarSketch- with the Computer Science Principles, a view of computing literacy that is emerging as a new standard for Advanced Placement and other high school computer science courses. The project is grounded on the premise that EarSketch, a STEM + Art (STEAM) learning environment, embodies authenticity (i.e., its cultural and industry relevance in both arts and STEM domains), along with a context that facilitates communication and collaboration among students (i.e., through a studio-based learning approach). These elements are critical to achieving successful outcomes across diverse student populations. Using agent-based modeling, the research team will investigate what factors enhance or impede implementation of authentic STEAM tools in different school settings.

The researchers will be engaged in a multi-stage process to develop: a) an implementation-ready, web-based EarSketch learning environment that integrates programming, digital audio workstation, curriculum, audio loop library, and social sharing features, along with studio-based learning functionality to support student presentation, critique, discussion, and collaboration; and b) an online professional learning course for teachers adopting EarSketch in Computer Science Principles courses. Using these resources, the team will conduct a quasi-experimental study of EarSketch in Computer Science Principles high school courses across the state of Georgia; measure student learning and engagement across multiple demographic categories; and determine to what extent an EarSketch-based CS Principles course promotes student achievement and engagement across different student populations. The project will include measures of student performance, creativity, collaboration, and communication in student programming tasks to determine the extent to which studio-based learning in EarSketch promotes success in these important areas. An agent-based modeling framework in multiple school settings will be developed to determine what factors enhance or impede implementation of EarSketch under conditions of routine practice.

Teaching STEM with Robotics: Design, Development, and Testing of a Research-based Professional Development Program for Teachers

Using design-based research, with teachers as design partners, the project will create and refine project-based, hands-on robotics curricula such that science and math content inherent in robotics and related engineering design practices are learned. To provide teachers with effective models to capitalize on robotics for elucidating science and math concepts, a design-based Professional Development program will be built using principles of technological, pedagogical, and content knowledge (TPACK).

Lead Organization(s): 
Award Number: 
1417769
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

Offering meaningful and motivating engineering contexts, such as robotics, within science and math courses constitutes a compelling strategy to address the Next Generation Science Standards and the Common Core State Standards for Math while enhancing science and math learning for all students. Using design-based research, with teachers as design partners, the project will create and refine project-based, hands-on curricula such that science and math content inherent in robotics and related engineering design practices are learned. To provide teachers with effective models to capitalize on robotics for elucidating science and math concepts, a design-based Professional Development program will be built using principles of technological, pedagogical, and content knowledge (TPACK). To ensure that teachers are well prepared, research-based practices and features of effective Professional Development will be adopted. Experts in robotics, engineering, education, curriculum design, and assessment--with experience in K-12 education, training, and outreach--have formed an interdisciplinary team to make robotics central to and sustainable in middle school science and math classrooms.

The research questions addressed in this project are qualitative in nature as appropriate for design research questions. The methodologies include teacher needs assessment, teachers' perceptions of robotics, pre and post testing, classroom observations, and surveys. Examples of the research questions are:

What characteristics of robotics promote effective learning of middle school science and math?

What elements of Professional Development engender teachers' TPACK of robotics and link it with classroom science and math?

What are student prerequisites to effectively use robotics in science and math learning?

What are the gains in students' STEM engagement, interest, persistence, and career awareness?

The robotics curriculum will include physical science used in robot performance expectations and motion stability. Additionally the curriculum will include the engineering design process consisting of problem definition, solution development, and design improvement. Robotics provides opportunities to support science and engineering practices of the Next Generation Science Standards such as developing and using models, planning and conducting investigations, designing solutions, and analyzing and interpreting data. The project will be aimed at middle school students and will provide substantial teacher professional development to implement the new curriculum modules. The partner schools have student bodies drawn from a diverse student population in New York City.

Pages

Subscribe to Assessment