Assessment

Assessing Secondary Teachers' Algebraic Habits of Mind (Collaborative Research: Stevens)

This collaborative project is developing instruments to assess secondary teachers' Mathematical Habits of Mind (MHoM). These habits bring parsimony, focus, and coherence to teachers' mathematical thinking and, in turn, to their work with students. This work fits into a larger research agenda with the ultimate goal of understanding the connections between secondary teachers' mathematical knowledge for teaching and secondary students' mathematical understanding and achievement.

Partner Organization(s): 
Award Number: 
1222496
Funding Period: 
Wed, 08/15/2012 to Sun, 07/31/2016
Full Description: 

Boston University, Education Development Center, Inc., and St. Olaf College are collaborating on Assessing Secondary Teachers' Algebraic Habits of Mind (ASTAHM) to develop instruments to assess secondary teachers' Mathematical Habits of Mind (MHoM). These habits bring parsimony, focus, and coherence to teachers' mathematical thinking and, in turn, to their work with students. MHoM is a critical component of mathematical knowledge for teaching at the secondary level. Recognizing the need for a scientific approach to investigate the ways in which MHoM is an indicator of teacher effectiveness, the partnership is researching the following questions:

1. How do teachers who engage MHoM when doing mathematics for themselves also bring MHoM to their teaching practice?

2. How are teachers' engagement with MHoM and their use of these habits in teaching related to student understanding and achievement?

To investigate these questions, ASTAHM is developing two instruments: a paper and pencil (P&P) assessment and an observation protocol that measure teachers' knowledge and classroom use, respectively, of MHoM.

The work is being conducted in two phases: (1) an instrument-refinement and learning phase, and (2) an instrument-testing and research phase. Objectives of Phase 1 are to gather data to refine the project's existing instruments and to learn about the bridge factors that impact the relationship between teachers' knowledge and classroom use of MHoM. Specific research activities include: administering the pilot P&P assessment to 40 teachers, videotaping Algebra instructions of 8 teachers, performing initial testing and refinement of the instruments, and using the data to analyze the bridge factors. Phase 2 is a large-scale study involving field-testing the P&P assessment with 200 teachers, videotaping 20 teachers and studying them using the observation protocol, collecting achievement data from 3000 students, and checking P&P content validity with 200 mathematicians. With these validated instruments in hand, the project will then conduct an investigation into the above research questions. Lesley University's Program Evaluation and Research Group (PERG) is the external evaluator. PERG is assessing ASTAHM's overall success in developing valid and reliable instruments to investigate the extent to which a relationship exists between teachers' MHoM and their classroom practice, as well as student achievement. Evaluators are also investigating whether users' coding guides for both instruments enable field-testers to effectively use and adequately score them.

This work fits into a larger research agenda with the ultimate goal of understanding the connections between secondary teachers' mathematical knowledge for teaching and secondary students' mathematical understanding and achievement. The MHoM construct is closely aligned with the Common Core State Standards-Mathematics (CCSS-M); especially its Standards for Mathematical Practice. For example, both place importance on seeking and using mathematical structure. Thus the instruments this project produces can act as pre- and post-measures of the effectiveness of professional development programs in preparing teachers to implement the CCSS-M. Mathematics teacher knowledge at the secondary level is an understudied field. Through analyses of the practices and habits of mind that teachers bring to their work, ASTAHM is developing instruments that can be used to shed light on effective secondary teaching.


Project Videos

2019 STEM for All Video Showcase

Title: Studying Teachers' Mathematical Habits of Mind

Presenter(s): Sarah Sword, Eden Badertscher, Al Cuoco, Miriam Gates, Ryota Matsuura, & Glenn Stevens

2017 STEM for All Video Showcase
Title: Assessing Secondary Teachers' Algebraic Habits of Mind

Presenter(s): Sarah Sword, Courtney Arthur, Al Cuoco, Miriam Gates, Ryota Matsuura, & Glenn Stevens

2016 STEM for All Video Showcase

Title: Assessing Secondary Teachers' Algebraic Habits of Mind

Presenter(s): Ryota Matsuura, Al Cuoco, Glenn Stevens, & Sarah Sword


Supporting the Emergence of a Professional Teaching Community Through Collective Knowledge-Building in Assessment and Feedback of Mathematical Thinking (Collaborative Research: Silverman)

This collaborative project is developing an online, professional teaching community that addresses issues of assessment in mathematics classes. The developers are building on the success of the NSF-supported Math Forum's Problem of the Week program to create a community that works to increase students' mathematics learning by helping teachers stimulate student thinking, assess that thinking, and provide useful feedback to students.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1222355
Funding Period: 
Sat, 09/01/2012 to Fri, 08/31/2018
Full Description: 

This collaborative work involves Drexel University and Temple University where they are developing an online, professional teaching community that is addressing issues of assessment in mathematics classes. The developers are building on the success of the NSF-supported Math Forum's Problem of the Week program to create a community that is working to increase students' mathematics learning by helping teachers stimulate student thinking, assess that thinking, and provide useful feedback to students. The teachers are working together to create rubrics for assessing the progress of students as they solve challenging mathematics problems. The program is structured so that the teachers are learning mathematics and assessment strategies in addition to establishing a research-based model for online, professional communities.

Researchers are studying how specific activities (e.g., discourse, active participation, use of rubrics, feedback, and reflection) and an online community support teachers' engagement in authentic and generative assessment. Researchers are using ethnographic methods to understand the development of the community, and conducting focus groups and individual interviews to determine the impact of participation in the community on mathematics teachers. In addition, they are collecting data through discourse analysis, student work analysis, and rubric analysis to determine the optimal design of the products. The intentional structure of the online community builds on research findings on creating professional communities and research on assessing mathematics learning.

Online professional teaching communities offer new venues for communication, professional development, and shared work among mathematics teachers. The Math Forum provides an optimal, online context for expanding the popular Problem of the Week into a productive discussion of assessment of problem solving, the building of specific rubrics, and the related reflection on how to encourage student thinking. This collaborative work will offer rubrics for assessing mathematical problem solving, a new model for online professional development, and extensive information on building an online mathematics community.

SimScientists Assessments: Physical Science Links

The goal of this project is to develop and validate a middle school physical science assessment strand composed of four suites of simulation-based assessments for integrating into balanced (use of multiple measures), large-scale accountability science testing systems. It builds on the design templates, technical infrastructure, and evidence of the technical quality, feasibility, and instructional utility of the NSF-funded Calipers II project. The evaluation plan addresses both formative and summative aspects.

Lead Organization(s): 
Award Number: 
1221614
Funding Period: 
Mon, 10/01/2012 to Fri, 09/30/2016
Full Description: 

The goal of this project is to develop and validate a middle school physical science assessment strand composed of four suites of simulation-based assessments for integrating into balanced (use of multiple measures), large-scale accountability science testing systems. It builds on the design templates, technical infrastructure, and evidence of the technical quality, feasibility, and instructional utility of the NSF-funded Calipers II project. The assessment strand consists of multilevel (increased thinking levels) assessment designs grounded on evidence-centered principles that target practices and key disciplinary conceptual schemes, such as matter, motion, energy, and waves identified in the National Research Council report "A Framework for K-12 Science Education: Practices, Crosscutting Knowledge, and Core Ideas". The assessment model vertically links simulations (interactive with feedback to students, coaching, and reflection); curriculum-embedded assessments for formative use; unit benchmark assessment for interim summative purposes; and a set of "signature tasks" (short-term simulations on recurring problem types). Members of the Advisory Board and an Assessment Review Panel actively participate in the development and implementation of this effort. Heller Research Associates is the external evaluator. The evaluation plan addresses both formative and summative aspects.

The project's theory of action is based on model-based learning and evidence-centered design reflective of the notion that the construct of science is multidimensional, requiring (a) understanding how the components of a science conceptual system interact to produce behaviors of the system; and (b) the use of inquiry practices to investigate the dynamic behaviors and underlying components' interactions of the system. A total of eight research and development questions guide the scope of work. The questions focus on: (a) validity (substantive and technical quality) of the individual simulation assessments; and (b) classroom implementation (feasibility, fidelity, utility). The methodology for test construction and revision follows the testing standards of major professional organizations (i.e., American Educational Research Association, American Psychological Association, and National Council of Measurement in Education) through three development phases. Phase I (Assessment Development) focuses on the alignment, quality, and prototype testing, including leverage and modification of prior work, and design of new assessment suites and signature tasks. Phase II (Pilot and Validation Studies) deals with the testing of all assessments, research instruments, and study methods. Phase III (Cross-Validation Studies) substantiates the multilevel integration assessment model, cross-validates the assessments piloted in Phase II, and establishes a reliable argument that the assessments measure the intended content and inquiry practices suitable for use in district and state-level assessment systems.

Expected outcomes are: (1) a research-informed and field-tested physical science simulations-based assessment model with high potential for extended use in middle school grades; and (2) a policy brief that provides recommendations for integrating assessments into districts and state large-scale, multi-level, balanced science assessments.

Cluster Randomized Trial of the Efficacy of Early Childhood Science Education for Low-Income Children

The research goal of this project is to evaluate whether an early childhood science education program, implemented in low-income preschool settings produces measurable impacts for children, teachers, and parents. The study is determining the efficacy of the program on Science curriculum in two models, one in which teachers participate in professional development activities (the intervention), and another in which teachers receive the curriculum and teachers' guide but no professional development (the control).

Project Email: 
Award Number: 
1119327
Funding Period: 
Mon, 08/15/2011 to Mon, 07/31/2017
Project Evaluator: 
Brian Dates, Southwest Counseling Services
Full Description: 

The research goal of this project is to evaluate whether an early childhood science education program, Head Start on Science, implemented in low-income preschool settings (Head Start) produces measurable impacts for children, teachers, and parents. The study is being conducted in eight Head Start programs in Michigan, involving 72 classrooms, 144 teachers, and 576 students and their parents. Partners include Michigan State University, Grand Valley State University, and the 8 Head Start programs. Southwest Counseling Solutions is the external evaluator.

The study is determining the efficacy of the Head Start on Science curriculum in two models, one in which 72 teachers participate in professional development activities (the intervention), and another in which 72 teachers receive the curriculum and teachers' guide but no professional development (the control). The teacher study is a multi-site cluster randomized trial (MSCRT) with the classroom being the unit of randomization. Four time points over two years permit analysis through multilevel latent growth curve models. For teachers, measurement instruments include Attitudes Toward Science (ATS survey), the Head Start on Science Observation Protocol, the Preschool Classroom Science Materials/Equipment Checklist, the Preschool Science Classroom Activities Checklist, and the Classroom Assessment Scoring System (CLASS). For students, measures include the "mouse house problem," Knowledge of Biological Properties, the physics of falling objects, the Peabody Picture Vocabulary Test-Fourth Edition, the Expressive Vocabulary Test-2, the Test of Early Mathematics Ability-3, Social Skills Improvement System-Rating Scales, and the Emotion Regulation Checklist. Measures for parents include the Attitudes Toward Science survey, and the Community and Home Activities Related to Science and Technology for Preschool Children (CHARTS/PS). There are Spanish versions of many of these instruments which can be used as needed. The external evaluation is monitoring the project progress toward its objectives and the processes of the research study.

This project meets a critical need for early childhood science education. Research has shown that very young children can achieve significant learning in science. The curriculum Head Start on Science has been carefully designed for 3-5 year old children and is one of only a few science programs for this audience with a national reach. This study intends to provide a sound basis for early childhood science education by demonstrating the efficacy of this important curriculum in the context of a professional development model for teachers.

An Examination of Science and Technology Teachers' Conceptual Learning Through Concept-Based Engineering Professional Development

This project will determine the viability of an engineering concept-based approach to teacher professional development for secondary school science teachers in life science and in physical science. The project refines the conceptual base for engineering at the secondary level learning to increase the understanding of engineering concepts by the science teachers. The hypothesis is that when teachers and students engage with engineering design activities their understanding of science concepts and inquiry are also enhanced.

Project Email: 
Award Number: 
1158615
Funding Period: 
Thu, 09/01/2011 to Mon, 10/31/2011
Project Evaluator: 
Karen Peterman
Full Description: 

Technology educators from Black Hills State University and Purdue University partner with science educators from the University of Massachusetts at Boston and Stevens Institute of Technology to determine the viability of an engineering concept-based approach to teacher professional development for secondary school science teachers in life science and in physical science. The project refines the conceptual base for engineering at the secondary level learning (previously developed by the PIs) to increase the understanding of engineering concepts by the science teachers. In a pilot test of two weeks of professional development with ten teachers from each discipline, teachers become familiar with engineering concepts and study the process of infusing engineering concepts into science curricula so that they can develop modules in their discipline to be taught during the following in the school year. The following summer the teachers debrief the process and develop additional modules for their discipline. The process is revised and repeated with 22 teachers from each discipline. Teachers are explicitly provided strategies to help them meet the needs of diverse learners. The outputs of this project include: 1) a preliminary framework for secondary level engineering education to be published in both research and practitioner journals; 2) a pilot tested and validated Engineering Concept Assessment; 3) engineering-infused curriculum modules in life and physical science; and 4) a professional development model to prepare science teachers to infuse engineering in their teaching.

The project compares student learning when particular concepts in physics and biology are taught through engineering design with learning the same concepts taught an earlier group of students with present reform techniques used in the discipline. The hypothesis is that when teachers and students engage with engineering design activities their understanding of science concepts and inquiry are also enhanced. The research component of the project employs an iterative design with the design of activities followed by development and implementation. An engineering concept assessment is developed and tested to examine teacher learning and to determine how engineering concepts can be infused into the science curricula for life and physical science. Other quantitative and qualitative instruments are developed to assess the teachers? understandings of the engineering concepts and their pedagogical implications.

There is increasing emphasis on integrative STEM education. New national and international assessments are developing engineering strands and emphasizing non-routine problem solving. The framework for the Next Generation Science Standards includes engineering as one of four strands. Stand alone engineering course are not likely to be widely used. This project develops engineering infused science units and determines the professional development needed to use them effectively.

Designing an Integrated Framework for Genetics Education to Develop Innovative Curricula and Assessments

This project is developing a model for integrating best practices in technology-supported instructional design and formative assessment for genetics instruction in upper elementary, middle and high school. Using the Web-based Inquiry Science Environment platform, the project is developing school curriculum that scaffold and model scientific practices, enable students to interface with real-world problems, provide opportunities for students to make connections between visible phenomena and underlying genetic processes, and promote student monitoring and reflection on learning.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1119055
Funding Period: 
Mon, 08/15/2011 to Tue, 07/31/2012
Full Description: 

Michigan State University is developing a model for integrating best practices in technology-supported instructional design and formative assessment for genetics instruction in upper elementary, middle and high school. The project partners with an urban school district in Texas and a suburban school district in Michigan. The objectives are: (1) to articulate a detailed standards- and research-base conceptual framework for describing students' conceptions of genetics and how students develop a full understanding of genetics across grade spans (upper elementary, middle and high school); (2) to develop innovative instructional materials and embedded assessments that provide richer information about students' conceptual understanding of genetics and help practitioners make decisions about what to do next in instruction; and (3) to examine the implementation of these instructional materials and assessments to investigate students' understanding of genetics concepts.

Using the Web-based Inquiry Science Environment (WISE) 4.0 platform (a technology-rich learning environment), the project is developing a 5-week elementary, middle, and secondary school curriculum models that scaffold and model scientific practices, enable students to interface with real-world problems, provide opportunities for students to make connections between visible phenomena and underlying genetic processes, and promote student monitoring and reflection on their learning. Each module will include animation- and stimulation-based contexts in WISE to provide rich occasions to press for building and developing reasoning and explanations. To promote teachers' use of student responses in formative ways, the materials will offer clear guidance about how to make evidence-based instructional decisions as well as provide options for contingent instruction activities that can be used to address persistent or common non-normative ways of reasoning.

The research offers generalizable approaches on the principled design of embedded assessments in WISE 4.0 and on using these assessments formatively. A quasi-experimental study employing a cross-sectional and longitudinal comparison design will investigate the development of students' understanding of genetics-related ideas from upper elementary to the high school years.

ScratchJr: Computer Programming in Early Childhood Education as a Pathway to Academic Readiness and Success (Collaborative Research: Bers)

This project is researching and developing a new version of the Scratch programming language to be called ScratchJr, designed specifically for early childhood education (K-2). This work will provide research-based evidence regarding young children's abilities to use an object-oriented programming language and to study the impact this has on the children's learning of scientific concepts and procedures.

 

Project Email: 
Lead Organization(s): 
Award Number: 
1118664
Funding Period: 
Mon, 08/01/2011 to Thu, 07/31/2014
Full Description: 

This collaborative project between Tufts University and the Massachusetts Institute of Technology is researching and developing a new version of the Scratch programming language to be called ScratchJr, designed specifically for early childhood education (K-2). The current version of Scratch, which is widely implemented, is intended for ages 8-16 and is not developmentally appropriate for young children. This work will provide research-based evidence regarding young children's abilities to use an object-oriented programming language and to study the impact this has on the children's learning of scientific concepts and procedures. The team will develop ScratchJr in an iterative cycle, testing it in both in the Devtech lab at Tufts and the Eliot Pearson lab school and with a wider network of early childhood partners. At the end of the three-year project, ScratchJr will have been tested with approximately 350 students in K-2, 40 parents, and 58 early childhood educators.

ScratchJr will have three components: 1) a developmentally appropriate interface, with both touch screen and keyboard/mouse options; 2) an embedded library of curricular modules with STEM content to meet federal and state mandates in early childhood education; and 3) an on-line resource and community for early childhood educators and parents. The research questions focus on whether ScratchJr can help these young children learn foundational knowledge structures such as sequencing, causality, classification, composition, symbols, patterns, estimation, and prediction; specific content knowledge; and problem solving skills.

This interdisciplinary proposal makes contributions to the fields of learning technologies, early childhood education and human computer interaction. ScratchJr has the potential for broad implementation in both formal and informal settings.

Examining Formative Assessment Practices for English Language Learners in Science Classrooms (Collaborative Research: Li)

This is an exploratory study to identify critical aspects of effective science formative assessment (FA) practices for English Language Learners (ELLs), and the contextual factors influencing such practices. FA, in the context of the study, is viewed as a process contributing to the science learning of ELLs, as opposed to the administration of discrete sets of instruments to collect data from students. The study targets Spanish-speaking, elementary and middle school students.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1118951
Funding Period: 
Thu, 09/01/2011 to Sat, 08/31/2013
Project Evaluator: 
Advisory board members
Full Description: 

This is a two-year exploratory study to identify critical aspects of effective science formative assessment (FA) practices for English Language Learners (ELLs), and the contextual factors influencing such practices. Three institutions join efforts for this purpose: University of Colorado at Boulder, University of Colorado at Denver, and University of Washington. FA, in the context of the study, is viewed as a process contributing to the science learning of ELLs, as opposed to the administration of discrete sets of instruments to collect data from students. The study targets Spanish-speaking, elementary and middle school students. Findings from this study contribute to advance knowledge and understanding of FA as an inherent component of the science learning process in linguistically diverse classrooms, and to define a research agenda aimed at enhancing science teachers' ability to enact equitable and effective assessment practices for this student subpopulation.

Three research questions guide the work: (1) What FA practices are occurring in science classrooms that serve predominantly mainstream students and in those serving predominantly ELLs?; (2) How are teachers' FA practices for mainstream students different from or similar to those used with ELLs?; and (3) How do contextual factors and teachers' cultural and linguistic competencies influence FA practices? To address these questions, two conceptual frameworks are used--one for characterizing FA events; the other for examining FA events as a communication process. The study employs a mixed-methods research approach with emphasis on case studies. The sample size consists of three school districts in Colorado and Washington, 16 classrooms (8 elementary, 8 middle school), 16 teachers, and 96 ELLs. Classrooms are selected to represent a particular combination of four factors: (a) teacher ethnicity, (b) teacher formal academic preparation in teaching ELLs, (c) type of linguistic student background, and (d) grade level. Students are selected through a stratified random sample, identified by achievement level (i.e., low, medium, high), and linguistic background (i.e., mainstream, ELL). Data collection strategies to document the implementation of FA at the beginning, during, and at the end of a science unit include: (a) classroom observation protocols, (b) classroom video-recording, (c) video/artifact simulated recall, (d) assessment artifacts, (e) student interviews, (f) teacher questionnaires, (g) teacher interviews, (h) school principal interviews, and (i) school observations. Reliability and validity of most of the data-gathering instruments is determined through pilot studies. Data interpretation strategies include: (a) coding based on the two conceptual frameworks, (b) scoring rubrics to identify levels of effectiveness, and (c) narratives and profiles to describe FA patterns. Publications and the development of a website constitute the main dissemination strategies. A technical advisory board is responsible for formative and summative evaluation. Key evaluation questions are: (1) To what extent does the project enhance research on ELL FA practices through case studies?, and (2) How effectively do the project dissemination activities facilitate understanding of FA practices?

Major project outcomes include: (1) a description of the patterns of formal and informal FA practices for ELLs; (2) a comparison of the FA practices observed in classrooms that vary on the dimensions of teacher characteristics and linguistic diversity; and (3) an empirically and theoretically informed set of findings and strategies for supporting teachers to enact and enhance FA practices sensitive to cultural and linguistic diversity. Three main products are developed: (1) a monograph describing the FA practices observed across the different classrooms with concrete examples; (2) a description of possible professional development strategies to improve in-service FA practices for linguistically diverse students; and (3) a research-informed approach for analyzing FA practices. Besides filling the existing research gap on FA with ELLs, outcomes and products serve as a foundation for a future research agenda and a comprehensive project aimed at ensuring equitable science learning for all students, including ELLs.

Developing Teaching Expertise in K-5 Mathematics

This project designs materials and an accompanying support system to enable the development of expertise in the teaching of mathematics at the elementary level. The project has four main components: online professional development modules; practice-based assessments; resources for facilitators; and web-based technologies to deliver module content to diverse settings. Three modules are being developed and focus on fractions, reasoning and explanation, and geometry. Each module is organized into ten 1.5 hour sessions.

Project Email: 
Award Number: 
1118745
Funding Period: 
Thu, 09/01/2011 to Fri, 08/31/2018
Project Evaluator: 
American Institutes for Research
Full Description: 

Developers and researchers at the University of Michigan and the University of Denver are engaged in a project to design materials and an accompanying support system to enable the development of expertise in the teaching of mathematics at the elementary level. The project has four main components: a set of online professional development modules; practice-based assessments; a set of resources for facilitators; and web-based technologies to deliver module content to diverse settings. Three modules are planned: one focused on fractions and one focused on reasoning and explanation designed by Deborah Ball, Hyman Bass and the University of Michigan development team; and one on geometry developed by Douglas Clements and Julie Sarama at the University of Denver. Each module is organized into ten 1.5 hour sessions. 

Each module goes through a two-year design and development process that includes initial design, piloting, revision, and dissemination. Modules are piloted in a variety of settings, including university based courses for practicing teachers and district based in-service activities. These contexts include face-to-face professional development, real-time distance learning, and combinations of the two. Data are collected on participant engagement with the modules, on teacher classroom practice, and on mathematical knowledge for teaching.

The modules and associated materials will be widely available and will be free to schools. The materials can be imported into any learning management system, such as Blackboard, Moodle, and others.

Leveling Up: Supporting and Measuring High School STEM Knowledge Building in Social Digital Games

This project designs, develops and tests a digital gaming environment for high school students that fosters and measures science learning within alternate reality games about saving Earth's ecosystems. Players work together to solve scientific challenges using a broad range of tools including a centralized web-based gaming site and social networking tools, along with handheld smart-phones, and an avatar-based massively multiplayer online environment. The game requires players to contribute to a scientific knowledge building community.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1119144
Funding Period: 
Fri, 07/01/2011 to Mon, 06/30/2014
Project Evaluator: 
New Knowledge Organization
Full Description: 

This project designs, develops and tests a digital gaming environment for high school students that fosters and measures science learning within alternate reality games about saving Earth's ecosystems. Players work together to solve scientific challenges using a broad range of tools including a centralized web-based gaming site and social networking tools, along with handheld smart-phones, and an avatar-based massively multiplayer online environment (MMO). EdGE at TERC joins with GameGurus, high school teachers and assessment specialists to develop Leveling Up. The game requires players to contribute to a scientific knowledge building community; and players rate each other's contributions for their value to the communities' learning and decision-making in solving the challenge. Designers also work with high-school teachers to develop bridge activities that leverage science learning in games for use in formal education. Overall, the project goal is to understand the potential of the gaming environment as a direct intervention and as a catalyst to transform and measure high school STEM learning.

The research on Leveling Up compares the science learning measured within social digital games to class-based assessments of similar content and skills and explains the results using data from design documents, participant observations, surveys, interviews and student work. Formative research and iterative design with a cohort of with 15 testbed classes (grades 10-12) result in a set of assessments that have been validated in terms of scientific constructs and a set of common equivalent curriculum and assessments for implementation studies. In the third year of the project, researchers study 12 treatment classes and 3 control classes to compare students' advancement in the game to their gains on classroom assessments. In addition, half of the testbed classes use the classroom bridge activities and half do not, yielding samples of 180 students for each treatment and 90 students for the control sample. Researchers use multilevel models to examine the impact of the Leveling Up game play and bridge activities on high-school students' science knowledge. Independent evaluators (ILI) validate the interpretation of findings from the formative and implementation research.

Leveling Up is a fundamental first step for the STEM education field to understand how the pervasive social media emerging in today's society, including the phenomena of social digital gaming, can be leveraged to create exciting and productive STEM learning environments for the future. These technologies and knowledge building processes are critical for building a workforce of tomorrow that is scientifically, technologically, and data literate and also embody the inquiry and collaboration skills to contribute to productive and informed decisions about Earth's ecosystems and other important scientific and societal issues of our times. The project, Leveling Up, results in an ongoing STEM gaming environment for the public as well as a model for high school STEM assessment that may be used in other social digital games. Finally, Leveling Up also contributes a model for activities that bridge scientific inquiry occurring in social digital games with skills and content taught in high school STEM classes.

Pages

Subscribe to Assessment