Assessment

CAREER: Making Science Visible: Using Visualization Technology to Support Linguistically Diverse Middle School Students' Learning in Physical and Life Sciences

Award Number: 
1552114
Funding Period: 
Wed, 06/01/2016 to Mon, 05/31/2021
Full Description: 

The growing diversity in public schools requires science educators to address the specific needs of English language learners (ELLs), students who speak a language other than English at home. Although ELLs are the fastest-growing demographic group in classrooms, many are historically underserved in mainstream science classrooms, particularly those from underrepresented minority groups. The significant increase of ELLs at public schools poses a challenge to science teachers in linguistically diverse classrooms as they try to support and engage all students in learning science. The proposed project will respond to this urgent need by investigating the potential benefits of interactive, dynamic visualization technologies, including simulations, animations, and visual models, in supporting science learning for all middle school students, including ELLs. This project will also identify design principles for developing such technology, develop additional ways to support student learning, and provide new guidelines for effective science teachers' professional development that can assist them to better serve students from diverse language backgrounds. The project has the potential to transform traditional science instruction for all students, including underserved ELLs, and to broaden their participation in science.

In collaboration with eighth grade science teachers from two low-income middle schools in North Carolina, the project will focus on three objectives: (1) develop, test, and refine four open-source, web-based inquiry units featuring dynamic visualizations on energy and matter concepts in physical and life sciences, aligned with the Next Generation Science Standards (NGSS); (2) investigate how dynamic visualizations can engage eighth-grade ELLs and native-English-speaking students in science practices and improve their understanding of energy and matter concepts; and (3) investigate which scaffolding approaches can help maximize ELLs' learning with visualizations. Research questions include: (1) Which kinds of dynamic visualizations (simulations, animations, visual models) lead to the best learning outcomes for all students within the four instructional science units?; (2) Do ELLs benefit more from visualizations (or particular kinds of visualizations) than do native-English-speaking students?; and (3) What kinds of additional scaffolding activities (e.g., critiquing arguments vs. generating arguments) are needed by ELLs in order to achieve the greatest benefit? The project will use design-based research and mixed-methods approaches to accomplish its research objectives and address these questions. Furthermore, it will help science teachers develop effective strategies to support students' learning with visualizations. Products from this project, including four NGSS-aligned web-based inquiry units, the visualizations created for the project, professional development materials, and scaffolding approaches for teachers to use with ELLs, will be freely available through a project website and multiple professional development networks. The PI will collaborate with an advisory board of experts to develop the four instructional units, visualizations, and scaffolds, as well as with the participating teachers to refine these materials in an iterative fashion. Evaluation of the materials and workshops will be provided each year by the advisory board members, and their feedback will be used to improve design and implementation for the next year. The advisory board will also provide summative evaluation of student learning outcomes and will assess the success of the teachers' professional development workshops.

An Intelligent Ecosystem for Science Writing Instruction

The ability to express scientific ideas in both written and oral form is an important 21st century skill. This project would help teachers help students achieve these skills through automating an effective feedback process, in ways that are customized to particular disciplines and local classroom needs, particularly in high needs districts. The project will contribute to knowledge about how students learn to write and how computer assisted systems can support this learning.

Lead Organization(s): 
Award Number: 
1416980
Funding Period: 
Mon, 09/01/2014 to Thu, 08/31/2017
Full Description: 

The ability to express scientific ideas in both written and oral form is an important 21st century skill. Teachers, employers, and college faculty lament the inability of many high school graduates to write clearly. This deficit in writing is due in part because teachers do not have the time to provide appropriate, timely feedback to students on their writing. This project would help teachers help students achieve these skills through automating an effective feedback process, in ways that are customized to particular disciplines and local classroom needs, particularly in high needs districts. The project will contribute to knowledge about how students learn to write and how computer assisted systems can support this learning.

This project will develop and test three tools: 1) Teaching resources organized as developmental trajectories for teachers to use (e.g. from more simple to more complex; with diagnostics and strategies for addressing particular challenges); 2) A teacher dashboard that uses Artificial Intelligence tools to provide timely formative assessment to teachers by highlighting problem areas in their students' writing and peer reviews; and 3) An online teacher resource exchange to rapidly grow the set of appropriate assignments that can be used with this approach, critically filtered by student performance metrics. The project builds on a current system called SWoRD, which supports student peer reviewing in many disciplines within and beyond science. Working with six lead teachers and larger set of pilot teachers, the project will develop a trajectory of effective writing assignments in Biology, Chemistry, and Physics. In year three, there will be a summative evaluation with 90 teachers.

Exploring Ways to Transform Teaching Practices to Increase Native Hawaiian Students' Interest in STEM

This project will integrate Native Hawaiian cross-cultural practices to explore ways to help teachers know about and know how to connect resources of students' familiar worlds to their science teaching. This project will transform the ways teachers orient their teaching at the upper elementary and middle grades through professional development courses offered at the University of Hawaii at Manoa.

Lead Organization(s): 
Award Number: 
1551502
Funding Period: 
Tue, 09/01/2015 to Fri, 08/31/2018
Full Description: 

This project will integrate Native Hawaiian cross-cultural practices to explore ways to help teachers know about and know how to connect resources of students' familiar worlds to their science teaching. This research is needed since Native Hawaiians are often stereotyped as poor learners; the available STEM workforce falls short of meeting the demands of STEM employers in the state; and as the largest group of public school enrollees, data show a greater decline in percent of students meeting or exceeding proficiency in science at higher grade levels. This project will address these issues by transforming the ways teachers orient their teaching at the upper elementary and middle grades through professional development courses offered at the University of Hawaii at Manoa.

The professional development model for teachers will be situated in the larger national and global contexts of an increasingly technology oriented, urbanized society with associated marginalization of indigenous people whose traditional ecological knowledge and indigenous languages are often overlooked. Guided by the cultural mental model theory and a mixed methods approach, data will be collected through document analysis, surveys, individual and focus group interviews, and pre-post assessments. This approach will capture initials findings about the influence of the professional development model on teaching and learning in science. The end products from this project will be an improved professional development model that is more sensitive to contexts that promote learning by Native Hawaiian students. It will also produce a survey instrument to assess student interest and engagement in science learning whose teachers will have participated in the professional development model being explored. Both outcomes will potentially be instrumental in changing the way approximately 2000 Native Hawaiian students learn about and become more interested in STEM fields through their natural world.

Building Assessment Items and Instructional Tasks to Build Intercommunity Capacity to Develop Teachers' Mathematical Knowledge for Teaching

The infrastructure to improve mathematics education in the US requires building human resources in mathematics and mathematics education into a professional community that can respond to the critical needs in the field. This project seeks to build a professional community with shared understanding of the specialized content knowledge (SCK) - the special forms and ways of reasoning about mathematical knowledge used in teaching (MKT). 

Lead Organization(s): 
Award Number: 
1502778
Funding Period: 
Tue, 09/01/2015 to Sat, 08/31/2019
Full Description: 

The infrastructure to improve mathematics education in the US requires building human resources in mathematics and mathematics education into a professional community that can respond to the critical needs in the field. This project seeks to build a professional community with shared understanding of the specialized content knowledge (SCK) - the special forms and ways of reasoning about mathematical knowledge used in teaching (MKT). This community will help increase the capacity for further research and development on teachers' SCK in mathematics, which has been shown to relate to student achievement. Building on the professional community's shared knowledge, the project will also work to collaboratively develop an item bank of MKT/SCK items and tasks using the platform developed by the Illustrative Mathematics group for similar task and item development for K-12 students. Better measures, with a larger item bank, will help support both the learning and assessment of teachers' MKT/SCK.

Based on theories of communities of practice, this project will bring together mathematicians and mathematics educators to build a professional community with a shared understanding of the SCK in mathematics through engaging in efforts to develop items to measure SCK and the development of a task bank. Based on the "item camps" they engaged with to develop prior measures, the project will host twelve 4-day camps with varying themes for pairs of faculty and teachers or graduate students, ensuring a mix of mathematicians and educators. This work will lead to a certification for MKT/SCK. They will also develop approximately 60 items and have a mechanism for the future review and publishing of items. Using case-study methodologies, the project will study the development of these partnerships and the professional communities within and across the camps.

Quality Urban Ecology Science Teaching for Diverse Learners

This project will examine the relationship between teacher professional development associated with newly developed modules in urban ecology and the achievement and engagement of long-term English learners (LTEL).  Existing Urban Ecology learning modules will be enhanced to accommodate the needs of LTELs, and teachers will participate in professional development aimed at using the new materials to effectively integrate academic science discourse and literacy development for LTELs.

Lead Organization(s): 
Award Number: 
1503519
Funding Period: 
Sat, 08/01/2015 to Fri, 05/31/2019
Full Description: 

The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This exploratory research project will examine the relationship between teacher professional development associated with newly developed modules in urban ecology and the achievement and engagement of long-term English learners (LTEL). Participants in the project will include students in grades 4-8 in a large urban school district, elementary school teachers, middle school science teachers, and middle school teachers of English language arts. Existing Urban Ecology learning modules will be enhanced to accommodate the needs of LTELs, and teachers will participate in professional development aimed at using the new materials to effectively integrate academic science discourse and literacy development for LTELs.

The project will develop two enhanced urban ecology modules (47 lessons) for English learners in grades 4-8; science language and literacy assessments for English language learners (ELLs); an ELL STEM career awareness inventory; an urban ecology for ELLs teacher knowledge scale, and an urban ecology for ELLs pedagogy observation protocol. The materials will be tested with a stratified random sample of students identified by achievement level (low, medium, and high) and linguistic background (mainstream, LTEL, and "at risk" of becoming LTEL). A mixed-methods research design will be used to test the hypothesis that the quantity and quality of LTEL science language and literacy achievement will increase as a result of teacher participation in implementing the newly developed transdisciplinary framework for Urban Ecology for English Learners.

Playing with the Data: Developing Digital Supports for Middle School Science Teachers using Game-based Formative Assessment

This project will use cycles of design-based research to build new knowledge about how to facilitate teachers' interpretation and use of digital game-based formative assessment data. The research will also inform the revision and expansion of Playfully, an existing, online data-reporting dashboard that can be used with multiple digital games.

Award Number: 
1503255
Funding Period: 
Wed, 07/01/2015 to Sat, 06/30/2018
Full Description: 

The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

This project will use cycles of design-based research to build new knowledge about how to facilitate teachers' interpretation and use of digital game-based formative assessment data. The research will also inform the revision and expansion of Playfully, an existing, online data-reporting dashboard that can be used with multiple digital games. The project is a collaboration between researchers at Education Development Center Inc.'s Center for Children and Technology (EDC|CCT) and the assessment and game development teams at GlassLab. The research and development teams will engage in a three-year partnership with 60 middle-grade science teachers working in diverse school settings in different parts of the country. The aim of the project is to refine an online formative assessment platform that utilizes data from a video game designed to teach argumentation at the middle school level. It provides rigorous research on the design features of data tools and associated materials available to teachers to inform their ongoing instruction (i.e., formative assessment tools) when using game-based platforms.

Dissemination of the results of this project will include practical, evidence-based suggestions for supporting middle school science teachers' use of digital games for assessment, and for the design and implementation of data dashboards. Key audiences include educational game designers, game-based assessment developers, formative assessment experts, and leaders in middle grade science teaching and learning.

Student-Adaptive Pedagogy for Elementary Teachers: Promoting Multiplicative and Fractional Reasoning to Improve Students' Preparedness for Middle School Mathematics

The project develops a teacher professional development intervention to support student-adaptive pedagogy for multiplicative and fractional reasoning. The idea is that classroom instruction should build on students' current conceptions and experiences. It focuses on students from urban, underserved and low-socioeconomic status populations who often fall behind in the elementary grades and are left underprepared for middle grades mathematics.

Lead Organization(s): 
Award Number: 
1503206
Funding Period: 
Wed, 07/15/2015 to Sun, 06/30/2019
Full Description: 

The project develops a teacher professional development intervention to support student-adaptive pedagogy for multiplicative and fractional reasoning. The idea is that classroom instruction should build on students' current conceptions and experiences. The context for the study is grades 3-5 teachers in Aurora Public Schools. It focuses on students from urban, underserved and low-socioeconomic status populations who often fall behind in the elementary grades and are left underprepared for middle grades mathematics. It includes a summer workshop and academic year follow-up including teacher collaboration. The project provides tools for capitalizing on successful, school-based research for promoting teachers' buy-in, adoption, and sustaining of student-adaptive pedagogy. The project also includes measurement of student understanding of the concepts. An extensive plan to share tools and resources for teachers and instructional coaches (scalable to district/state levels) and of research instruments and findings, will promote sharing project outcomes with a wide community of stakeholders (teachers, administrators, researchers, parents, policy makers) responsible for students' growth. This is a Full Design & Development project within the DRK-12 Program's Learning Strand. The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

The project aims to implement and study a professional development intervention designed to shift upper-elementary teachers' mathematics teaching toward a constructivist approach, called student-adaptive pedagogy (AdPed), which adapts teaching goals and activities based on students' conceptions and experiences. The project focuses on multiplicative and fractional reasoning--critical for students' success in key areas of middle school mathematics (e.g., ratio, proportion, and function). The project seeks to design an instrument for measuring teachers' implementation of AdPed, a clinical interview rubric for students' multiplicative reasoning and then an analysis of teachers' content knowledge and the implementation of AdPed following the professional development. The research design is rooted in an innovative, cohesive framework that integrates four research-based components: (i) a model of mathematics learning and knowing, (ii) models of progressions in students' multiplicative and fractional reasoning, (iii) a model of teaching (AdPed) to promote such learning, and (iv) a mathematics teacher development continuum. Capitalizing on successful preliminary efforts in the Denver Metro area to refine a PD intervention and student-adaptive tools that challenge and transform current practices, the project will first validate and test instruments to measure (a) teacher growth toward adaptive pedagogy and (b) students' growth in multiplicative reasoning. Using these new instruments, along with available measures, the project will then promote school-wide teacher professional development (grades 3-5) in multiple schools in an urban district with large underserved student populations and study the professional development benefits for teacher practices and student outcomes. The mixed methods study includes classroom-based data (e.g., video analysis, lesson observations, teacher interviews) and measures of students' multiplicative reasoning specifically and mathematical understanding generally.

Zoombinis: The Full Development Implementation Research Study of a Computational Thinking Game for Upper Elementary and Middle School Learners

This project leverages an existing game by embedding tools for studying patterns of students' decision-making and problem solving in the environment. This allows researchers to understand how students learn about computational thinking within a tool that bridges informal and formal learning settings to engage a wide variety of students. The project will also develop tools and resources for classroom teachers.

Lead Organization(s): 
Award Number: 
1502882
Funding Period: 
Wed, 07/15/2015 to Sat, 06/30/2018
Full Description: 

The Logical Journey of the Zoombinis implementation research study examines the development of computational thinking for upper elementary and middle grades students. Computational thinking is the set of ideas and practices considered vital for computer science skills and has been attracting increased attention over the past several years in K-12 education. This project leverages an existing game by embedding tools for studying patterns of students' decision-making and problem solving in the environment. This allows researchers to understand how students learn about computational thinking within a tool that bridges informal and formal learning settings to engage a wide variety of students. The project will also develop tools and resources for classroom teachers. The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

The research examines three questions. First, what strategies do players develop during Zoombinis gameplay that may provide evidence of implicit computational thinking? Second, how can teachers leverage implicit knowledge of computational thinking developed in Zoombinis to improve formal (explicit) learning? Third, how can a large-scale commercial game be used for broad and equitable improvement of computational thinking? The research uses and develops educational data mining techniques to assess students' learning in conjunction with pre-post computational thinking assessments (external to the game), teacher interviews, classroom observations, and case studies of classroom use. The goal is to understand both students' learning of computational thinking and how to bridge the formal and informal learning via classroom implementation of the Zoombinis game.

Strengthening the Quality, Design and Usability of Simulations as Assessments of Teaching Practice

Ensuring that beginning teachers are "classroom-ready" requires assessments that efficiently and validly evaluate proficiency in teaching. This project explores assessments involving simulated students as a way to assess teaching practice, which could provide an important complement, or alternative, to directly assessing teaching practice in classrooms.

Lead Organization(s): 
Award Number: 
1502711
Funding Period: 
Tue, 09/01/2015 to Thu, 08/31/2017
Full Description: 

Ensuring that beginning teachers are "classroom-ready" requires assessments that efficiently and validly evaluate proficiency in teaching. This project explores assessments involving simulated students as a way to assess teaching practice, which could provide an important complement, or alternative, to directly assessing teaching practice in classrooms. This form of assessment has the potential to provide a way to avoid onerous expense, logistics, and other difficulties of assessments happening in classrooms. The project will address questions about the development of performance expectations for elementary mathematics teachers, the extent to which the performance of the "student" role can be standardized across different performance contexts, and different approaches for generating teaching scenarios. The assessments will focus on the teaching practices of eliciting and interpreting students' mathematical thinking. The project will support: (1) establishing the validity of the assessment as a means to assess readiness to teach elementary mathematics and (2) providing the necessary foundation for scaling research and the use of simulation assessments. 

The goal of this project is generating, calibrating, and studying standardized simulations of clinical performance of mathematics teaching. The strategy is to investigate three components of the simulation assessment that will enable its broader use in the field. One component will focus on approaches that use different foundations (wisdom of practice, interactions with children, and learning trajectories research) for the design of simulations that are authentic and provide robust information about teaching. Data on the ways in which each approach supplies resources needed for assessment development will be compared. Another component will focus on the degree to which the role of the student can be standardized given the dynamics of teaching. Data on the responses of standardized students, who have similar initial training, to different situational categories will be analyzed. A final component will be establishing a basis for calibrating performance expectations for simulations linked to key points in a teacher's career trajectory (early career teachers, experienced teachers, "accomplished" teachers). Data on the performance of teachers at different points in their careers on the same assessment simulations will be compared. This study of components impacting assessment design will result in a more robust foundation for further development of, and further research on, teaching simulation assessments. The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

SimScientists Games: Development of Simulation-Based Game Designs to Enhance Formative Assessment and Deep Science Learning in Middle School

This project will focus on understanding how educational games, designed according to research-based learning and assessment design principles, can better assess and promote students' science knowledge, application of science process skills, and motivation and engagement in learning.

Lead Organization(s): 
Award Number: 
1503481
Funding Period: 
Sat, 08/01/2015 to Wed, 07/31/2019
Full Description: 

The Discovery Research K-12 (DRK-12) program seeks to significantly enhance the learning and teaching of science, technology, engineering, and mathematics (STEM) by preK-12 students and teachers through research and development of innovative resources, models, and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This project is a four-year design and development study submitted to the assessment strand of the program. It will focus on understanding how educational games, designed according to research-based learning and assessment design principles, can better assess and promote students' science knowledge, application of science process skills, and motivation and engagement in learning. The project will develop a new genre of games to serve as formative assessment resources designed to collect evidence of science learning during gameplay, provide feedback and coaching in the form of hints, and reinforce middle grade (6th-8th) students' life science concepts and investigation practices about ecosystems described in the Next Generation Science Standards (NGSS) (Achieve, 2013). The games will build on the designs of the simulation-based, curriculum-embedded assessments developed in previous NSF-funded efforts, which include student progress reports and reflection activities that allow teachers to provide feedback to students and adjust instruction. The design of the games will draw from multiple lines of research, such as cognition, particularly model-based learning; principled assessment design; and motivation. Intended to provide engaging activities for understanding and investigating the system components, roles, interactions, and population dynamics of ecosystems, the project will produce two sets of comprehensive games: (1) Organisms and Interactions, and (2) Emergent Population Levels: Managing an Ecosystem. Each game will consist of progressively advanced mini-games. Twenty-four California Bay Area middle school teachers will participate in the study. Teacher professional development (PD) will include face-to-face sessions and an online platform that permits a wide range of interactions among participants and the facilitators. The PD will emphasize the alignment of the ecosystem simulation-based curriculum modules with their state standards, instructional materials, and the new games. 

The project will address six research questions: (1) How well do the games align with the ecosystem crosscutting concepts, core ideas, and inquiry practices in the NGSS?; (2) How well do game components meet quality standards?; (3) How well do the games integrate with the existing simulation-based curriculum modules and the teachers' existing instructional sequence?; (4) What effect does the use of the games have on students' understanding of the science concepts, scientific practices, and collaboration skills?; (5) How does success in gameplay relate to improved performance on the external outcome measures comprised of the simulation-based benchmark and the pre/posttest?; and (6) How does the use of the games affect students' engagement in science learning? In a Year 1 usability study, the project will test, analyze, and revise alpha versions of the games. In Year 2, a classroom feasibility study of beta versions will inform further revisions. In Year 3, six teachers will pilot-test the games. A second pilot test in Year 4 will examine the effectiveness of the games by comparing student performance in classes using the existing simulation-based curriculum-embedded assessments and reflection activities with classes using the curriculum-embedded assessments plus the new games. Data collection and analysis strategies include: (a) alignment reviews; (b) focus groups and usability testing; (c) cognitive labs for construct validity and usability; (d) game reports (badges); (e) pre/posttest of American Association for the Advancement of Science (AAAS) items; (f) benchmark assessment data; (g) student interest in the games and science; (h) teacher surveys; (i) case studies; (j) game quality analysis; (k) differential item functioning; (l) analysis of covariance; and (m) analysis of variance on posttest scores (outcome variable) to compare the means across student groups (by intervention mode) and their prior science achievement levels.

Pages

Subscribe to Assessment