Learning Progression

Assessing College-Ready Computational Thinking (Collaborative Research: Wilson)

The goal of this project is to develop learning progressions and assessment items targeting computational thinking. The items will be used for a test of college-ready critical reasoning skills and will be integrated into an existing online assessment system, the Berkeley Assessment System Software.

Award Number: 
2010314
Funding Period: 
Tue, 09/01/2020 to Sat, 08/31/2024
Full Description: 

Because of the growing need for students to be college and career ready, high-quality assessments of college readiness skills are in high demand. To realize the goal of preparing students for college and careers, assessments must measure important competencies and provide rapid feedback to teachers. It is necessary to go beyond the limits of multiple-choice testing and foster the skills and thinking that lie at the core of college and career ready skills, such as computational thinking. Computational thinking is a set of valuable skills that can be used to solve problems, design systems, and understand human behavior, and is thus essential to developing a more STEM-literate public. Computational thinking is increasingly seen as a fundamental analytical skill that everyone, not just computer scientists, can use. The goal of this project is to develop learning progressions and assessment items targeting computational thinking. The items will be used for a test of college-ready critical reasoning skills and will be integrated into an existing online assessment system, the Berkeley Assessment System Software.

The project will address a set of research questions focused on 1) clarifying computational thinking constructs, 2) usability, reliability of validity of assessment items and the information they provide, 3) teachers' use of assessments, and 4) relationships to student performance. The study sample of 2,700 used for the pilot and field tests will include all levels of students in 10th through 12th grade and first year college students (both community college and university level). The target population is students in schools which are implementing the College Readiness Program (CRP) of the National Mathematics and Science Institute. In the 2020-21 academic year 54 high schools across 11 states (CA, GA, FL, ID, LA, NC, NM, OH, TX, VA, and WA) will participate. This will include high school students in Advanced Placement classes as well as non-Advanced Placement classes.  The team will use the BEAR Assessment System to develop and refine assessment materials. This system is an integrated approach to developing assessments that seeks to provide meaningful interpretations of student work relative to cognitive and developmental goals. The researchers will gather empirical evidence to develop and improve the assessment materials, and then gather reliability and validity evidence to support their use. In total, item response data will be collected from several thousand students. Student response data will be analyzed using multidimensional item response theory models.

Exploring Early Childhood Teachers' Abilities to Identify Computational Thinking Precursors to Strengthen Computer Science in Classrooms

This project will explore PK-2 teachers' content knowledge by investigating their understanding of the design and implementation of culturally relevant computer science learning activities for young children. The project team will design a replicable model of PK-2 teacher professional development to address the lack of research in early computer science education.

Lead Organization(s): 
Award Number: 
2006595
Funding Period: 
Tue, 09/01/2020 to Thu, 08/31/2023
Full Description: 

Strengthening computer science education is a national priority with special attention to increasing the number of teachers who can deliver computer science education in schools. Yet computer science education lacks the evidence to determine how teachers come to think about computational thinking (a problem-solving process) and how it could be integrated within their day-to-day classroom activities. For teachers of pre-kindergarten to 2nd (PK-2) grades, very little research has specifically addressed teacher learning. This oversight challenges the achievement of an equitable, culturally diverse, computationally empowered society. The project team will design a replicable model of PK-2 teacher professional development in San Marcos, Texas, to address the lack of research in early computer science education. The model will emphasize three aspects of teacher learning: a) exploration of and reflection on computer science and computational thinking skills and practices, b) noticing and naming computer science precursor skills and practices in early childhood learning, and c) collaborative design, implementation and assessment of learning activities aligned with standards across content areas. The project will explore PK-2 teachers' content knowledge by investigating their understanding of the design and implementation of culturally relevant computer science learning activities for young children. The project includes a two-week computational making and inquiry institute focused on algorithms and data in the context of citizen science and historical storytelling. The project also includes monthly classroom coaching sessions, and teacher meetups.

The research will include two cohorts of 15 PK-2 teachers recruited from the San Marcos Consolidated Independent School District (SMCISD) in years one and two of the project. The project incorporates a 3-phase professional development program to be run in two cycles for each cohort of teachers. Phase one (summer) includes a 2-week Computational Making and Inquiry Institute, phase two (school year) includes classroom observations and teacher meetups and phase three (late spring) includes an advanced computational thinking institute and a community education conference. Research and data collection on impacts will follow a mixed-methods approach based on a grounded theory design to document teachers learning. The mixed-methods approach will enable researchers to triangulate participants' acquisition of new knowledge and skills with their developing abilities to implement learning activities in practice. Data analysis will be ongoing, interweaving qualitative and quantitative methods. Qualitative data, including field notes, observations, interviews, and artifact assessments, will be analyzed by identifying analytical categories and their relationships. Quantitative data includes pre to post surveys administered at three-time points for each cohort. Inter-item correlations and scale reliabilities will be examined, and a repeated measures ANOVA will be used to assess mean change across time for each of five measures. Project results will be communicated via peer-reviewed journals, education newsletters, annual conferences, family and teacher meetups, and community art and culture events, as well as on social media, blogs, and education databases.

Paving the Way for Fractions: Identifying Foundational Concepts in First Grade (Collaborative Research: Newcombe)

The goal of this project is to investigate the extent to which individual differences in informal fraction-related knowledge in first-grade children are associated with short- and longer-term fractions and math outcomes, and to see whether there is a causal link between level of informal fraction-related knowledge and the ability to profit from fractions instruction that directly builds on this knowledge.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
2000424
Funding Period: 
Mon, 06/01/2020 to Fri, 05/31/2024
Full Description: 

Although fractions represent a crucial topic in early childhood education, many students develop only a tenuous grasp of fraction concepts, even after several years of fraction instruction that is aligned with current standards. The goal of this project, led by a team of researchers at the University of Delaware and Temple University, is to answer important questions about the informal understandings of fractions young children have before they come to school and what their relations are to fraction learning in more formal instructional settings. Proficiency with fractions dramatically increases the likelihood of students succeeding in math, which in turn increases participation in the STEM workforce. Importantly, large individual differences in fraction understandings are apparent at the start of fractions instruction in the intermediate grades. Early fraction misunderstandings cascade into more severe math weaknesses in later grades, especially when instruction may shift abruptly from whole numbers to fractions. There is a critical need to understand the roots of individual differences that arise before formal instruction takes place. Young children possess important informal fraction understandings before they come to school, but the range of these abilities and their role in formal fraction learning and development is not well understood. The goal of this project is: a) to investigate the extent to which individual differences in informal fraction-related knowledge in first-grade children are associated with short- and longer-term fractions and math outcomes; and b) to see whether there is a causal link between level of informal fraction-related knowledge and the ability to profit from fractions instruction that directly builds on this knowledge. The findings from the project hold promise for informing early childhood educators how fractions can be incorporated in the first-grade curriculum in new and meaningful ways. Though the findings should be beneficial to all students, the project will specifically target members of groups underrepresented in STEM fields, including ethnic and racial minority and low-income students.

The project design includes both an observational study and an experimental study. The observational study will: (1) document individual differences in informal fraction-related knowledge in first grade; (2) determine concurrent relations between this informal knowledge and general cognitive and whole number competencies; and (3) examine whether informal fraction-related knowledge at the beginning of first grade uniquely predicts math outcomes at the end. The experimental study will explore the extent to which first graders' informal and formal fraction concepts can be affected by training. The researchers will test whether training on the number line, which is continuous and closely aligned with the mental representation of the magnitude of all real numbers, will help students capitalize on their informal fraction understandings of proportionality, scaling, and equal sharing as well as their experience with integers to learn key fraction concepts. Together, the synergistic studies will pinpoint the role informal fraction knowledge in learning key fraction concepts. All data will be collected in Delaware schools serving socioeconomically and ethnically diverse populations of students. Primary measures include assessments of informal fraction knowledge (proportional reasoning, spatial scaling, equal sharing), executive functioning, vocabulary, whole number knowledge, whole number/fraction number line estimation, formal fraction knowledge, and broad mathematics achievement (calculation, fluency, applied problems).

Evolving Minds: Promoting Causal-Explanatory Teaching and Learning of Biological Evolution in Elementary School

Adopting a teaching and curricular approach that will be novel in its integration of custom explanatory storybook materials with hands-on investigations, this project seeks to promote third grade students' understanding of small- and large-scale evolution by natural selection. By studying students across multiple school districts, this research will shed light on the benefits to diverse students of instruction that focuses on supporting children's capacities to cogently explain aspects of the biological world rather than learn disparate facts about it.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
2009176
Funding Period: 
Mon, 06/01/2020 to Fri, 05/31/2024
Full Description: 

Natural selection is a fundamental mechanism of evolution, the unifying principle of biology. It is central to understanding the functional specialization of living things, the origin of species diversity and the inherent unity of biological life. Despite the early emergence of tendencies that can make evolution increasingly challenging to learn over time, natural selection is currently not taught until middle or high school. This is long after patterns of misunderstanding are likely to have become more entrenched. The current research responds to this situation. It targets elementary school as the time to initiate comprehensive instruction on biological evolution. Adopting a teaching and curricular approach that will be novel in its integration of custom explanatory storybook materials with hands-on investigations, it seeks to promote third grade students' understanding of small- and large-scale evolution by natural selection. By studying students across multiple school districts, this research will shed light on the benefits to diverse students of instruction that focuses on supporting children's capacities to cogently explain aspects of the biological world rather than learn disparate facts about it. It will also illuminate the value of simple tools, like explanatory storybooks, for elementary school teachers who are often expected to teach counterintuitive topics such as natural selection while not feeling confident in their own understanding.

This project will investigate changes in Grade 3 students' learning and reasoning about living things during implementation of a guided inquiry curriculum unit on evolution by natural selection that emphasizes causal-mechanistic explanation. Classroom inquiry activities and investigations into a range of real-world phenomena will be framed by engagement with a sequence of innovative custom causal-explanatory storybook, animation and writing prompt materials that were developed under prior NSF support to promote transferable, scientifically accurate theory- and evidence-based reasoning about natural selection. In response to the distinctive challenges of life science and evolution learning, the project will integrate and thematically unify currently disparate Next Generation Science Standards (NGSS) content and practice standards to create a comprehensive unit that addresses all three NGSS dimensions and is accompanied by evidence-based approaches to teacher professional development (PD). Using a design based research approach, and informed by cognitive developmental findings, this 4-year project will engage at least 700 students and their teachers and include partners from at least four school districts, Boston University, and TERC.

Developing and Investigating Unscripted Mathematics Videos

This project will use an alternative model for online videos to develop video units that feature the unscripted dialogue of pairs of students. The project team will create a repository of 6 dialogic mathematics video units that target important Algebra 1 and 2 topics for high school and upper middle school students, though the approach can be applied to any STEM topic, for any age level.

Lead Organization(s): 
Award Number: 
1907782
Funding Period: 
Sun, 09/01/2019 to Thu, 08/31/2023
Full Description: 

This project responds to the recent internet phenomenon of widespread accessibility to online instructional videos, which offer many benefits, such as student control of the pace of learning. However, these videos primarily focus on a single speaker working through procedural problems and providing an explanation. While the immense reach of free online instructional videos is potentially transformative, this potential can only be attained if access transcends physical availability to also include entry into important disciplinary understandings and practices, and only if the instructional method pushes past what would be considered outdated pedagogy in any other setting than a digital one. This project will use an alternative model for online videos, originally developed for a previous exploratory project, to develop 6 video units that feature the unscripted dialogue of pairs of students. The project team will use the filming and post-production processes established during the previous grant to create a repository of 6 dialogic mathematics video units that target important Algebra 1 and 2 topics for high school and upper middle school students, though the approach can be applied to any STEM topic, for any age level. They will also conduct 8 research studies to investigate the promise of these unscripted dialogic videos with a diverse population to better understand the vicarious learning process, which refers to learning from video- or audio-taped presentations of other people learning. Additionally, the project team will provide broader access to the project videos and support a variety of users, by: (a) subtitling the videos and checking math task statements for linguistic accessibility; (b) representing diversity of race, ethnicity, and language in both the pool of students who appear in the videos and the research study participants; (c) providing teachers with an array of resources including focus questions to pose in class with each video, printable task worksheets, specific ways to support dialogue about the videos, and alignment of the video content with Common Core mathematics standards and practices; and (d) modernizing the project website and making it functional across a variety of platforms.

The videos created for this project will feature pairs of students (called the talent), highlighting their unscripted dialogue, authentic confusion, and conceptual resources. Each video unit will consist of 7 video lessons (each split into 4-5 short video episodes) meant to be viewed in succession to support conceptual development over time. The project will build upon emerging evidence from the exploratory grant that as students engage with videos that feature peers grappling with complex mathematics, they can enter a quasi-collaborative relationship with the on-screen talent to learn complex conceptual content and engage in authentic mathematical practices. The research focuses on the questions: 1. What can diverse populations of vicarious learners learn mathematically from dialogic videos, and how do the vicarious learners orient to the talent in the videos? 2. What is the nature of vicarious learners' evolving ways of reasoning as they engage with multiple dialogic video lessons over time and what processes are involved in vicarious learning? and, 3. What instructional practices encourage a classroom community to adopt productive ways of reasoning from dialogic videos? To address the first question, the project team will conduct two Learning Outcomes and Orientation Studies, in which they analyze students' learning outcomes and survey responses after they have learned from one of the video units in a classroom setting. Before administering an assessment to a classroom of students, they will first conduct an exploratory Interpretation Study for each unit, in which they link the mathematical interpretations that VLs generate from viewing the project videos with their performance on an assessment instrument. Both types of studies will be conducted twice, once for each of two video units - Exponential Functions and Meaning and Use of Algebraic Symbols. For the second research question, the project team will identify a learning trajectory associated with each of four video units. These two learning trajectories will inform the instructional planning for the classroom studies by identifying what meaningful appropriation can occur, as well as conceptual challenges for VLs. By delivering learning trajectories for two additional units, the project can contribute to vicarious learning theory by identifying commonalities in learning processes evident across the four studies. For the final research question, the project team will investigate how instructors can support students with the instrumental genesis process, which occurs through a process called instrumental orchestration, as they teach the two videos on exponential functions and algebraic symbols.

Learning Trajectories as a Complete Early Mathematics Intervention: Achieving Efficacies of Economies at Scale

The purpose of this project is to test the efficacy of the Learning and Teaching with Learning Trajectories (LT2) program with the goal of improving mathematics teaching and thereby increasing young students' math learning. LT2 is a professional development tool and a curriculum resource intended for teachers to be used to support early math instruction and includes the mathematical learning goal, the developmental progression, and relevant instructional activities.

Lead Organization(s): 
Award Number: 
1908889
Funding Period: 
Mon, 07/01/2019 to Sun, 06/30/2024
Full Description: 

U.S. proficiency in mathematics continues to be low and early math performance is a powerful predictor of long-term academic success and employability. However, relatively few early childhood degree programs have any curriculum requirements focused on key mathematics topics. Thus, teacher professional development programs offer a viable and promising method for supporting and improving teachers' instructional approaches to mathematics and thus, improving student math outcomes. The purpose of this project is to test the efficacy of the Learning and Teaching with Learning Trajectories (LT2) program with the goal of improving mathematics teaching and thereby increasing young students' math learning. LT2 is a professional development tool and a curriculum resource intended for teachers to be used to support early math instruction. The LT2 program modules uniquely include the mathematical learning goal, the developmental progression, and relevant instructional activities. All three aspects are critical for high-quality and coherent mathematics instruction in the early grades.

This project will address the following research questions: 1) What are the medium-range effects of LT2 on student achievement and the achievement gap? 2) What are the short- and long-term effects of LT2 on teacher instructional approach, beliefs, and quality? and 3) How cost effective is the LT2 intervention relative to the original Building Blocks intervention? To address the research questions, this project will conduct a multisite cluster randomized experimental design, with 90 schools randomly assigned within school districts to either experimental or control groups. Outcome measures for the approximately 250 kindergarten classrooms across these districts will include the Research-based Elementary Math Assessment, observations of instructional quality, a questionnaire focused on teacher beliefs and practices, in addition to school level administrative data. Data will be analyzed using multi-level regression models to determine the effect of the Learning Trajectories intervention on student learning.

Case Studies of a Suite of Next Generation Science Instructional, Assessment, and Professional Development Materials in Diverse Middle School Settings

This project addresses a gap between vision and implementation of state science standards by designing a coordinated suite of instructional, assessment and teacher professional learning materials that attempt to enact the vision behind the Next Generation Science Standards. The study focuses on using state-of-the-art technology to create an 8-week long, immersive, life science field experience organized around three investigations.

Lead Organization(s): 
Award Number: 
1907944
Funding Period: 
Mon, 07/01/2019 to Fri, 06/30/2023
Full Description: 

New state science standards are ambitious and require important changes to instructional practices, accompanied by a coordinated system of curriculum, assessment, and professional development materials. This project addresses a gap between vision and implementation of such standards by designing a coordinated suite of instructional, assessment and teacher professional learning materials that attempt to enact the vision behind the Next Generation Science Standards. The study focuses on the design of such materials using state-of-the-art technology to create an 8-week long, immersive, life science field experience organized around three investigations. Classes of urban students in two states will collect data on local insect species with the goal of understanding, sharing, and critiquing environmental management solutions. An integrated learning technology system, the Learning Navigator, draws on big data to organize student-gathered data, dialogue, lessons, an assessment information. The Learning Navigator will also amplify the teacher's role in guiding and fostering next generation science learning. This project advances the field through an in-depth exploration of the goals for the standards documents. The study begins to address questions about what works when, where, and for whom in the context of the Next Generation Science Standards.

The project uses a series of case studies to create, test, evaluate and refine the system of instructional, assessment and professional development materials as they are enacted in two distinct urban school settings. It is designed with 330 students and 22 teachers in culturally, racially and linguistically diverse, under-resourced schools in Pennsylvania and California. These schools are located in neighborhoods that are economically challenged and have students who demonstrate patterns of underperformance on state standardized tests. It will document the process of team co-construction of Next Generation Science-fostering instructional materials; develop assessment tasks for an instructional unit that are valid and reliable; and, track the patterns of use of the instructional and assessment materials by teachers. The study will also record if new misconceptions are revealed as students develop Next Generation Science knowledge,  comparing findings across two diverse school locations in two states. Data collection will include: (a) multiple types of data to establish validity and reliability of educational assessments, (b) the design, evaluation and use of a classroom observation protocol to gather information on both frequency and categorical degree of classroom practices that support the vision, and (c) consecutive years of ten individual classroom enactments through case studies analyzed through cross-case analyses. This should lead to stronger and better developed understandings about what constitutes strong Next Generation Science learning and the classroom conditions, instructional materials, assessments and teacher development that foster it.

Aligning the Science Teacher Education Pathway: A Networked Improvement Community

This project will study the activities of a Networked Improvement Community (NIC) as a vehicle to bridge gaps across four identified steps along the science teacher training and development pathways within local contexts of 8 participating universities. The overarching goal of the project is to strengthen the capacity of universities and school districts to reliably produce teachers of science who are knowledgeable about and can effectively enact the Next Generation Science Standards (NGSS), although prepared in varied organizational contexts.

Award Number: 
1908900
Funding Period: 
Mon, 07/01/2019 to Fri, 06/30/2023
Full Description: 

California State University will study the activities of a Networked Improvement Community (NIC) as a vehicle to bridge gaps across four identified steps along the science teacher training and development pathways within local contexts of 8 participating universities (NIC sites). Networked Improvement Community (NIC) will co-create a shared vision and co-defined research agenda between university researchers, science educators and school district practitioners working together to reform teacher education across a variety of local contexts. By studying outcomes of shared supports and teacher tools for use in multiple steps along the science teacher education pathway, researchers will map variation existing in the system and align efforts across the science teacher education pathway. This process will integrate an iterative nature of educational change in local contexts impacting enactment of the NGSS in both university teacher preparation programs and in school district professional training activities and classrooms.

The overarching goal of the project is to strengthen the capacity of universities and school districts to reliably produce teachers of science who are knowledgeable about and can effectively enact the Next Generation Science Standards (NGSS), although prepared in varied organizational contexts. The project will accomplish this goal 1) leveraging the use of an established Networked Improvement Community, composed of science education faculty from eight university campuses and by 2) improving and studying coherence in the steps along the science teacher education pathway within and across these universities and school districts. The project will use a mixed methods approach to data collection and analysis. Consistent with Improvement Science Theory, research questions will be co-defined by all stakeholders.

CAREER: Black Youth Development and Curricular Supports for Robust Identities in Mathematics

This study seeks to describe trajectories that describe the ways in which Black learners develop as particular kinds of mathematical learners. The study takes place in the context of an established, multi-year college bridge program that has as its goals to increase the representation of historically marginalized groups in the university community.

Lead Organization(s): 
Award Number: 
1845841
Funding Period: 
Wed, 05/01/2019 to Tue, 04/30/2024
Full Description: 

Student success in mathematics correlates with positive identities, dispositions, and relationships towards the subject. As mathematics education research strives to understand historic inequities in mathematics for Black learners, small-scale research has described the relationships between identity, subjectivity, and positionality in Black learners as it relates to their achievement and interest in mathematics. This study builds on that descriptive work by seeking to describe trajectories that describe the ways in which Black learners develop as particular kinds of mathematical learners. The study takes place in the context of an established, multi-year college bridge program that has as its goals to increase the representation of historically marginalized groups in the university community. Students in the bridge program from three communities in the greater Detroit area with strong academic achievement in mathematics will be recruited. Their experiences in the bridge program will be traced to identify trajectories that describe the development of Black learners relative to mathematics, and document the design features of classroom activities that support learners in moving through those trajectories.

At the center of the project is the study of cohorts of students in grades 8-11 as they move through the summer bridge program. The bridge project's current curriculum features a series of lessons focused on identity development related to mathematics. These lessons will be implemented, studied, revised, and redeployed across the duration of the project across the summer sessions. Teacher focus groups and surveys will assess the implementation of the activities and aggregate feedback on the design. Three cohorts of students will be recruited to participate in the broader project activities from three metro areas with distinctly different demographic profiles. Student mathematical efficacy will be assessed for all participating students. Within each of the three metro areas, students will be recruited that represent differing levels of mathematics efficacy to ensure that focus students are likely to experience different trajectories through their engagement with the study. The students will be interviewed three times in each academic year to describe their trajectories. Student achievement data will also be collected for all participating students along with narrative descriptions and autobiographies about the messages students receive about mathematics. These messages include their own internal thinking about how they see themselves as mathematics learners, and messages that are sent to them by other students, teachers, and the community. Products of the study will be case studies that describe trajectories of identity development in Black mathematics learners, and a disseminated curriculum for a mathematics identity-focused bridge program supporting Black learners.

Engaging High School Students in Computer Science with Co-Creative Learning Companions (Collaborative Research: Magerko)

This research investigates how state-of-the-art creative and pedagogical agents can improve students' learning, attitudes, and engagement with computer science. The project will be conducted in high school classrooms using EarSketch, an online computer science learning environments that engages learners in making music with JavaScript or Python code.

Award Number: 
1814083
Funding Period: 
Sat, 09/15/2018 to Wed, 08/31/2022
Full Description: 
This research investigates how state-of-the-art creative and pedagogical agents can improve students' learning, attitudes, and engagement with computer science. The project will be conducted in high school classrooms using EarSketch, an online computer science learning environments that engages over 160,000 learners worldwide in making music with JavaScript or Python code. The researchers will build the first co-creative learning companion, Cai, that will scaffold students with pedagogical strategies that include making use of learner code to illustrate abstraction and modularity, suggesting new code to scaffold new concepts, providing help and hints, and explaining its decisions. This work will directly address the national need to develop computing literacy as a core STEM skill.
 
The proposed work brings together an experienced interdisciplinary team to investigate the hypothesis that adding a co-creative learning companion to an expressive computer science learning environment will improve students' computer science learning (as measured by code sophistication and concept knowledge), positive attitudes towards computing (self-efficacy and motivation), and engagement (focused attention and involvement during learning). The iterative design and development of the co-creative learning companion will be based on studies of human collaboration in EarSketch classrooms, the findings in the co-creative literature and virtual agents research, and the researchers' observations of EarSketch use in classrooms. This work will address the following research questions: 1) What are the foundational pedagogical moves that a co-creative learning companion for expressive programming should perform?; 2) What educational strategies for a co-creative learning companion most effectively scaffold learning, favorable attitudes toward computing, and engagement?; and 3) In what ways does a co-creative learning companion in EarSketch increase computer science learning, engagement, and positive attitudes toward computer science when deployed within the sociocultural context of a high school classroom? The proposed research has the potential to transform our understanding of how to support student learning in and broaden participation through expressive computing environments.

Pages

Subscribe to Learning Progression