Qualitative

Development of the Electronic Test of Early Numeracy

The project will develop and refine an electronic Test of Early Numeracy (e-TEN) in English and Spanish that will assess informal and formal knowledge of number and operations in domains including verbal counting, numbering, numerical relationships, and mental addition/subtraction. The overarching goal of the assessment design is to create a measure that is more accurate, more accessible to a wider range of children, and easier to administer than existing measures.

Partner Organization(s): 
Award Number: 
1621470
Funding Period: 
Thu, 09/15/2016 to Tue, 08/31/2021
Full Description: 

The project will develop and refine an electronic Test of Early Numeracy (e-TEN) in English and Spanish, focused on number and operations. The assessment will incorporate a learning trajectory that describes students' development of the understanding of number. The electronic assessment will allow for the test to adapt to students' responses and incorporate games to increase children's engagement with the tasks. These features take advantage of the electronic format. The achievement test will be designed to be efficient, user-friendly, affordable, and accessible for a variety of learning environments and a broad age range (3 to 8 years old). The overarching goal of the assessment design is to create a measure that is more accurate, more accessible to a wider range of children, and easier to administer than existing measures. This project is funded by the Discovery Research Pre-K-12 Program, which funds research and development of STEM innovations and approaches in assessment, teaching and learning.

The e-TEN will assess informal and formal knowledge of number and operations in domains including verbal counting, numbering, numerical relationships, and mental addition/subtraction. The items will be designed using domain-based learning trajectories that describe students' development of understanding of the topics. The test will be designed with some key characteristics. First, it will be semi-adaptive over six-month age spans. Second, it will have an electronic format that allows for uniform implementation and an efficient, user-friendly administration. The test will also be accessible to Spanish speakers using an inclusive assessment model. Finally, the game-based aspect should increase children's engagement and present more meaningful questions. The user-friendly aspect includes simplifying the assessment process compared to other tests of numeracy in early-childhood. The first phase of the development will test a preliminary version of the e-TEN to test its functionality and feasibility. The second phase will focus on norming of the items, reliability and validity. Reliability will be assessed using Item Response Theory methods and test-retest reliability measures. Validity will be examined using criterion-prediction validity and construct validity. The final phase of the work will include creating a Spanish version of the test including collecting data from bilingual children using both versions of the e-TEN.

Algebra Project Mathematics Content and Pedagogy Initiative

This project will scale up, implement, and assess the efficacy of interventions in K-12 mathematics education based on the well-established Algebra Project (AP) pedagogical framework, which seeks to improve performance and participation in mathematics of students in distressed school districts, particularly low-income students from underserved populations.

Award Number: 
1621416
Funding Period: 
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description: 

Algebra continues to serve as a gatekeeper and potential barrier for high school students. The Algebra Project Mathematics Content and Pedagogy Initiative (APMCPI) will scale up, implement and assess the efficacy of interventions in K-12 mathematics education based on the well-established Algebra Project (AP) pedagogical framework. The APMCPI project team is comprised of four HBCUs (Virginia State University, Dillard University, Xavier University, Lincoln University), the Southern Initiative Algebra Project (SIAP), and four school districts that are closely aligned with partner universities. The purpose of the Algebra Project is to improve performance and participation in mathematics by members of students in distressed school districts, particularly those with a large population of low-income students from underserved populations including African American and Hispanics. The project will provide professional development and implement the Algebra Project in four districts and study the impact on student learning. The research results will inform the nation's learning how to improve mathematics achievement for all children, particularly those in distressed inner-city school districts.

The study builds on a prior pilot project with a 74% increase in students who passed the state exam. In the early stages of this project, teachers in four districts closely associated with the four universities will receive Algebra Project professional development in Summer Teacher Institutes with ongoing support during the academic year, including a community development plan. The professional development is designed to help teachers combine mathematical problem solving with context-rich lessons, which both strengthen and integrate teachers' understanding of key concepts in mathematics so that they better engage their students. The project also will focus on helping teachers establish a framework for mathematically substantive, conceptually-rich and experientially-grounded conversations with students. The first year of the study will begin a longitudinal quasi-experimental, explanatory, mixed-method design. Over the course of the project, researchers will follow cohorts who are in grade-levels 5 through 12 in Year 1 to allow analyses across crucial transition periods - grades 5 to 6; grades 8 to 9; and grades 12 to college/workforce. Student and teacher data will be collected in September of Project Year 1, and in May of each project year, providing five data points for each student and teacher participant. Student data will include student attitude, belief, anxiety, and relationship to mathematics and science, in addition to student learning outcome measures. Teacher data will include content knowledge, attitudes and beliefs, and practices. Qualitative data will provide information on the implementation in both the experimental and control conditions. Analysis will include hierarchical linear modeling and multivariate analysis of covariance.

Sonified Interactive Simulations for Accessible Middle School STEM

For this project, researchers will iteratively develop simulations to include sonifications, non-speech sounds that represent visual information, aimed at enhancing accessibility for all learners, but particularly for those with visual impairments to produce sonified simulations, professional development resources, design guidelines and exemplars, and publications.

Lead Organization(s): 
Award Number: 
1621363
Funding Period: 
Sat, 10/01/2016 to Mon, 09/30/2019
Full Description: 

Students with disabilities often have fewer opportunities for experiential learning, an important component of quality STEM education. Computer based simulations in science can provide valuable opportunities for these students to experience and manipulate natural phenomena related to critical STEM ideas. However, existing simulations remain largely inaccessible to students with visual impairments in particular. Recent advances in technology related to sonification use with simulations can make it possible for these students to have a more complete and authentic experience. Sonification is the use of non-speech sounds, such as musical tones, to represent visual information including data. Such sounds can be manipulated temporally and spatially and can also vary by amplitude and frequency to convey information that is more traditionally displayed visually. 

Researchers will iteratively develop five middle school physical science simulations to include sonifications aimed at enhancing accessibility for all learners, but particularly for those with visual impairments. Data collection activities will include focus groups and interviews with students and teachers focused on engagement. The end products of this project will include sonified simulations, professional development resources, design guidelines and exemplars, and publications.

Proportions Playground: A Dynamic World to Support Teachers' Proportional Reasoning

This project focuses on the creation of the initial functionality for a dynamic microworld, Proportions Playground, designed to support teachers in developing a coherent understanding of proportional reasoning. The Proportions Playground project seeks to both develop a unique pilot software application for the iPad and explore how it supports teachers in developing a coherent, robust definition of proportions.

Award Number: 
1621290
Funding Period: 
Thu, 09/01/2016 to Thu, 02/28/2019
Full Description: 

Proportions are a critical topic in mathematics that is simultaneously complicated and over-simplified in typical instruction. Current research undertaken by the research team suggests that the over-simplification is related to limitations in teachers' understandings of proportional relationships. Presenting proportions in a dynamic environment offers teachers the opportunity to create key developmental understandings related to this area of mathematics. This project focuses on the creation of the initial functionality for a dynamic microworld, Proportions Playground, designed to support teachers in developing a coherent understanding of proportional reasoning. Proportions Playground is conceptualized as a tool for supporting the development of coherent understandings by allowing teachers to interact in concrete ways with otherwise abstract ideas and by allowing teachers easy access to dynamic objects and other representations. It is meant to address the significant limitations for reasoning about the relationships between measurable aspects of two objects as well as in manipulating those relationships. Building from work currently underway, Proportions Playground will explore key areas in which there are opportunities for engaging teachers in the development of a coherent and robust understanding of proportional reasoning that extends beyond the typical "3 given, 1 unknown" proportion problem. This approach attempts to engage teachers in an array of dynamic, visually-rich sets of tasks designed to challenge teachers' preconceptions of proportions and to strengthen their connections between proportions and related areas of mathematics. This project is funded by the Discovery Research PreK-12 (DRK-12) and EHR Core Research (ECR) Programs. the DRK-12 program supports research and development on STEM education innovations and approaches to teaching, learning, and assessment. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field.

The Proportions Playground project seeks to both develop a unique pilot software application for the iPad and explore how it supports teachers in developing a coherent, robust definition of proportions. The software will be designed to support either numeric manipulation (e.g., graphing software) or geometric constructions (e.g., dynamic geometry software). Specifically, for this project the mathematics of interest will include the relationships between similarity and proportion and the nature of covariation. The research will focus on how teachers are developing a robust and coherent understanding of proportions and how the dynamic environment promotes such understandings. Working with six teacher advisors, the project will develop three task sets. Using teaching experiments and individual interviews, results will be used to refine the task sets. The revised task sets will be piloted with 40 teachers. Data will be collected on participants' thinking and any changes seen in the knowledge resources they are using. The researchers will be looking for factors that seem to impact teachers' thinking as well as evidence to support or deny the assertion that the Proportions Playground activities engage teachers in (a) different ways of reasoning about proportions and (b) support them in drawing from a wide array of resources so that coherence may be developed were the teachers to have a prolonged engagement with the tools. The project will rely on Epistemic Network Analysis to identify the connections between knowledge resources.

Zoom In! Learning Science with Data

This project will address the need for high quality evidence-based models, practices, and tools for high school teachers and the development of students' problem solving and analytical skills by leveraging novel research and design approaches using digital tools and two well-established online instructional platforms: Zoom In and Common Online Data Analysis Platform.            

Award Number: 
1621289
Funding Period: 
Thu, 09/15/2016 to Sat, 08/31/2019
Full Description: 

This project will expand the DRK-12 portfolio by contributing to a limited program portfolio on data science, and also by being responsive to a broader, national discourse on data science, exemplified in the data-dependent scientific practices emphasis in the Next Generation Science Standards (NGSS). With the impetus toward data literacy, an acute need has emerged for high quality evidence-based models, practices, and tools to better prepare high school teachers to teach data skills and for students to develop the problem solving and analytical skills needed to interpret and understand data, particularly in the sciences. This project will address these challenges by leveraging novel research and design approaches, using digital tools and two well-established online instructional platforms; Zoom In and Common Online Data Analysis Platform.

With a user base of over 27,000 teachers and students, the existing Zoom In platform has proven successful in fostering evidence-based inquiry among social studies teachers. This project will test the feasibility of the platform to facilitate data-focused inquiry and skill development among high school science teachers and their students. In Year 1, two NGSS-aligned digital curriculum modules and supporting materials focused on scientific phenomena and problems in biology and earth science will be developed for high school science teachers and embedded in a modified iteration of Zoom In. The Common Online Data Analysis Platform (CODAP) will be integrated into the modules to make hierarchical data structures, modeling, visualizations, and dynamic linking possible within Zoom In. A pilot and usability test will be conducted with 16 teachers and 100 students from diverse New York City public high schools. Two teacher focus groups and think-aloud sessions with the students will be held. In Year 2, the remaining four modules will be developed. Guided by four research questions, field testing with teachers and students will be done to assess the content, CODAP data tools, Zoom-in student supports, teacher supports, and outcome measures. In Year 3, final revisions to the tools will be completed. A small-scale efficacy test will be conducted to assess aspects of the implementation process, practices, and overall impact of the modules on student learning. For the efficacy study, a two-level cluster-randomized design will be employed to randomly assign schools to the Zoom In intervention. A comparison group will use another existing well-designed data literacy digital instructional platform but without key aspects of Zoom In. Outcome measures will be administered at the beginning and end of the school year to the treatment and comparison groups. Back-end data, observational data, and teacher log data will be collected and analyzed. Qualitative data will be gathered from teacher and student observations and interviews and analyzed. Researchers will analyze the impact on student learning using hierarchical linear models with an effect treatment condition and student-and-class-level covariates. The research findings will be broadly disseminated through the Zoom In platform, conferences, publications, and social media.


Project Videos

2019 STEM for All Video Showcase

Title: Zoom In! Learning Science with Data

Presenter(s): Megan Silander & Bill Tally


Building a Next Generation Diagnostic Assessment and Reporting System within a Learning Trajectory-Based Mathematics Learning Map for Grades 6-8

This project will build on prior funding to design a next generation diagnostic assessment using learning progressions and other learning sciences research to support middle grades mathematics teaching and learning. The project will contribute to the nationally supported move to create, use, and apply research based open educational resources at scale.

Award Number: 
1621254
Funding Period: 
Thu, 09/15/2016 to Sat, 08/31/2019
Full Description: 

This project seeks to design a next generation diagnostic assessment using learning progressions and other research (in the learning sciences) to support middle grades mathematics teaching and learning. It will focus on nine large content ideas, and associated Common Core State Standards for Mathematics. The PIs will track students over time, and work within school districts to ensure feasibility and use of the assessment system.

The research will build on prior funding by multiple funding agencies and address four major goals. The partnership seeks to address these goals: 1) revising and strengthening the diagnostic assessments in mathematics by adding new item types and dynamic tools for data gathering 2) studying alternative ways to use measurement models to assess student mathematical progress over time using the concept of learning trajectories, 3) investigating how to assist students and teachers to effectively interpret reports on math progress, both at the individual and the class level, and 4) engineering and studying instructional strategies based on student results and interpretations, as they are implemented within competency-based and personalized learning classrooms. The learning map, assessment system, and analytics are open source and can be used by other research and implementation teams. The project will exhibit broad impact due to the number of states, school districts and varied kinds of schools seeking this kind of resource as a means to improve instruction. Finally, the research project contributes to the nationally supported move to create, use, and apply research based open educational resources at scale.

Understanding and Improving Learning from Online Mathematics Classroom Videos

The purpose of this project is to investigate issues in the design and implementation of effective virtual learning communities (VLCs) for teachers and to examine the relation between teachers' reflective engagement with VLCs and their students' mathematics learning outcomes. Findings from this project will be used to build and share effective ways to support teacher learning online.

Award Number: 
1621253
Funding Period: 
Mon, 08/15/2016 to Fri, 07/31/2020
Full Description: 

U.S. elementary teachers face many challenges. They are asked to teach all subjects to students with different needs and abilities. To do this well, they need good professional learning opportunities. Many teachers look online for such opportunities, but little is known about the quality of those opportunities and how they can be improved to help teachers meet their challenges. The goals of this project are to learn more about how teachers use one popular website for elementary mathematics teachers and how this website and similar ones can be adapted to better support teacher learning. Specifically, the project will (1) interview teachers about their use of the website, (2) investigate how to improve the ways teachers interact with video resources on the site by testing different ways of guiding their attention, and (3) examine how teachers' interactions with these video resources are related to their students' learning of mathematics. Findings from this project will be used to build and share effective ways to support teacher learning online. The project will thus benefit teachers who use the popular website, teachers who use similar websites, researchers who study how teachers learn from such websites, and the students of teachers who learn from such websites.

Video-based learning has been the focus of much professional development research over the past decade. As video-based learning has been found to be effective, many professional developers have taken this learning to scale through the online space. A number of high profile and popular virtual learning communities (VLCs) have emerged to allow teachers to interact with video, but the scant number of studies on the effectiveness of such VLCs show some difficulties in engaging teachers in sustained, reflective professional learning. The purpose of this project is to investigate several major issues in the design and implementation of effective VLCs for teachers and to examine the relation between teachers' reflective engagement with VLCs and their students' mathematics learning outcomes. The investigators propose 3 studies, which build on each other, to address these issues. This project will (1) interview teachers who are members of a popular VLC, to investigate what they learn and how they contribute to community; (2) investigate conditions that impact the posting of reflective commentary about video cases through iterative experiments, as reflective commentary has the potential to build community and to support teacher learning; and (3) investigate the relation between reflective reactions to video cases and student mathematics outcomes. Through these investigations, this project will explore issues that impact the scalability of teachers learning asynchronously from online video. Results will be used to develop guided pathways - a prototype of an innovation that will be based on the results from the research - on one widely used VLC. Thus, this project will provide both a contribution to the field of STEM teacher education research and an immediate, research-based product that can be disseminated to thousands of teachers through an existing VLC.


Project Videos

2019 STEM for All Video Showcase

Title: Influencing Online Teacher Reflection

Presenter(s): Meg Bates, Cheryl Moran, & Michelle Perry

2018 STEM for All Video Showcase

Title: Exploring Teacher Learning from Online Lesson Video

Presenter(s): Meg Bates & Genevieve Henricks


Learning Evolution through Human and Non-Human Case Studies

This project will develop and test two curriculum units on the topic of evolution for high school general biology courses, with one unit focusing primarily on human case studies to teach evolution and one unit focusing primarily on case studies of evolution in other species. The two units will be compared to examine how different approaches to teaching evolution affect students and teachers.

Lead Organization(s): 
Award Number: 
1621194
Funding Period: 
Thu, 09/15/2016 to Tue, 08/31/2021
Full Description: 

This project aligns with Alabama's College & Career-Ready Standards (CCRS) for biology in grades 9-12 relating to Unity and Diversity. These standards are based on the Next Generation Science Standards (NGSS) and go into effect during the 2016-2017 school year. Building on prior work (DRL-119468), this project will develop and test two curriculum units on the topic of evolution for high school general biology courses, with one unit focusing primarily on human case studies to teach evolution and one unit focusing primarily on case studies of evolution in other species. The two units will be compared to examine how different approaches to teaching evolution affect students and teachers. The project will also develop and field test a Cultural and Religious Sensitivity (CRS) Resource to provide teachers with strategies for creating supportive learning environments where understanding of the scientific account of evolution is aided while also acknowledging the cultural controversy associated with learning about evolution. The impacts on student and teacher outcomes of using the curriculum units and the CRS Resource will be tested in classrooms by comparing the outcomes of the human versus non-human units, and by using or not using classroom strategies from the CRS Resource.

The project will examine student and teacher outcomes of four treatment groups: 1) Curriculum Unit 1, 2) Curriculum Unit 1 with the CRS Resource, 3) Curriculum Unit 2, and 4) Curriculum Unit 2 with the CRS Resource. The research questions are: 1) In what ways does using examples of human versus non-human evolution to teach core evolutionary concepts affect understanding of, acceptance of, and motivation to learn about evolution among high school introductory biology students? 2) In what ways do using teaching strategies that focus on acknowledging the cultural controversy about evolution using a procedural neutrality approach affect high school introductory biology teachers' comfort and confidence with teaching evolution? 3) In what ways does using examples of human versus non-human evolution to teach fundamental evolutionary concepts in conjunction with teaching strategies that focus on acknowledging the cultural controversy about evolution using a procedural neutrality approach affect understanding of, acceptance of, and motivation to learn about evolution among high school introductory biology students? And 4) In what ways does using examples of human versus non-human evolution to teach fundamental evolutionary concepts in conjunction with teaching strategies that focus on acknowledging the cultural controversy about evolution using a procedural neutrality approach affect high school introductory biology teachers' comfort and confidence with teaching evolution? The project will use a 2 X 2 X 2 mixed factorial quasi-experimental research design to answer these questions, and will include a total of 32 teachers, 8 in each treatment group, along with approximately 800 students. Each assessment will be administered as a pretest two weeks prior to starting the curriculum unit and as a posttest immediately after completing the unit. Test scores will be the within-subjects factors, and Curriculum Unit and CRS Resource will be the between-subjects factors.

Geological Models for Explorations of Dynamic Earth (GEODE): Integrating the Power of Geodynamic Models in Middle School Earth Science Curriculum

This project will develop and research the transformational potential of geodynamic models embedded in learning progression-informed online curricula modules for middle school teaching and learning of Earth science. The primary goal of the project is to conduct design-based research to study the development of model-based curriculum modules, assessment instruments, and professional development materials for supporting student learning of (1) plate tectonics and related Earth processes, (2) modeling practices, and (3) uncertainty-infused argumentation practices.

Lead Organization(s): 
Award Number: 
1621176
Funding Period: 
Mon, 08/15/2016 to Fri, 07/31/2020
Full Description: 

This project will contribute to the Earth science education community's understanding of how engaging students with dynamic computer-based systems models supports their learning of complex Earth science concepts regarding Earth's surface phenomena and sub-surface processes. It will also extend the field's understandings of how students develop modeling practices and how models are used to support scientific endeavors. This research will shed light on the role uncertainty plays when students use models to develop scientific arguments with model-based evidence. The GEODE project will directly involve over 4,000 students and 22 teachers from diverse school systems serving students from families with a variety of socioeconomic, cultural, and racial backgrounds. These students will engage with important geoscience concepts that underlie some of the most critical socio-scientific challenges facing humanity at this time. The GEODE project research will also seek to understand how teachers' practices need to change in order to take advantage of these sophisticated geodynamic modeling tools. The materials generated through design and development will be made available for free to all future learners, teachers, and researchers beyond the participants outlined in the project.

The GEODE project will develop and research the transformational potential of geodynamic models embedded in learning progression-informed online curricula modules for middle school teaching and learning of Earth science. The primary goal of the project is to conduct design-based research to study the development of model-based curriculum modules, assessment instruments, and professional development materials for supporting student learning of (1) plate tectonics and related Earth processes, (2) modeling practices, and (3) uncertainty-infused argumentation practices. The GEODE software will permit students to "program" a series of geologic events into the model, gather evidence from the emergent phenomena that result from the model, revise the model, and use their models to explain the dynamic mechanisms related to plate motion and associated geologic phenomena such as sedimentation, volcanic eruptions, earthquakes, and deformation of strata. The project will also study the types of teacher practices necessary for supporting the use of dynamic computer models of complex phenomena and the use of curriculum that include an explicit focus on uncertainty-infused argumentation.

Supporting Success in Algebra: A Study of the Implementation of Transition to Algebra

The project will research the implementation of Transition to Algebra, a year-long mathematics course for underprepared ninth grade students taken concurrently with Algebra 1 to provide additional support, and its impact on students' attitudes and achievement in mathematics in combination with teachers' instruction and the types of supports teachers need to successfully implement the intervention.

Partner Organization(s): 
Award Number: 
1621011
Funding Period: 
Sat, 10/01/2016 to Wed, 09/30/2020
Full Description: 

The project will research the impact and implementation of Transition to Algebra, a year-long mathematics course for underprepared ninth grade students taken concurrently with Algebra 1 to provide additional support. Nationally, there is a need for programs that support students' learning of algebra and that provide research-based resources and models particularly for students in need of additional support. The design of the Transition to Algebra curriculum reflects the idea that students underprepared for Algebra 1 can benefit from very specific help in building the logic of algebra by connecting arithmetic pattern and algebraic structure. The materials feature the use of mental mathematics, puzzles, explorations, and student dialogues to connect arithmetic pattern to algebraic structure. These features should encourage students to expect mathematics ideas to make sense, and to build algebraic habits of mind and problem solving stamina. The research will investigate the effects of the curriculum on students' algebra achievement and their attitudes towards mathematics. The Discovery Research PreK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

The research questions examine the impact of the Transition to Algebra course on students' attitudes and achievement in mathematics in combination with teachers' instruction and the types of supports teachers need to successfully implement the intervention. The project will use a pre-post quasi-experimental design, along with propensity score methods to reduce selection bias threats, to examine the implementation in approximately 35 treatment schools and 35 comparison schools. Qualitative and quantitative data will be collected and analyzed to address research questions. The study will also investigate how teachers use and adapt Transition to Algebra materials, and the supports critical to successful implementation. For example, the study will examine whether and how school and district activities such as common planning time, coaching, and other professional development experiences influence the implementation fidelity of the curriculum. Qualitative data will be collected through interviews and classroom observations. Quantitative data will be collected using student and teacher surveys, an algebra readiness assessment, a standardized end-of-course assessment, and students' scores on state tests.

Pages

Subscribe to Qualitative