Post-secondary Faculty

Validity Evidence for Measurement in Mathematics Education (V-M2ED) (Collaborative Research: Krupa)

The purpose of this project is to fully explore the mathematics education literature to synthesize what validity evidence is available for quantitative assessments in mathematics education.

Partner Organization(s): 
Award Number: 
1920619
Funding Period: 
Thu, 08/01/2019 to Wed, 07/31/2024
Full Description: 

As education has shifted more towards data-driven policy and research initiatives in the last several decades, data for policy-related aspects are often expected to be more quantitative in nature.  This has led to the increase in use of more quantitative measures in STEM education, including mathematics education. Unfortunately, evidence regarding the validity and reliability of mathematics education measures is lacking. Furthermore, the evidence for validity for quantitative tools and measures is not conceptualized or defined consistently by researchers in the field. The purpose of this project is to fully explore the mathematics education literature to synthesize what validity evidence is available for quantitative assessments in mathematics education. Drawing on the results of the synthesis study, the researchers will design, curate, and disseminate a repository of quantitative assessments used in mathematics education teaching and research. The researchers will also create materials and online training for a variety of scholars and practitioners to use the repository.

The team will address two main research questions: 1) How might validity evidence related to quantitative assessments used in mathematics education research be categorized and described? and 2) What validity evidence exists for quantitative instruments used in mathematics education scholarship since 2000? Researchers will use a cross-comparative methodology which involves conducting a literature search and then analyzing and categorizing features of instruments. The research team will examine cases (i.e., assessments described in manuscripts) in which quantitative instruments have been used, alongside specific features such as the construct measured, evidence related to sources of validity, and study sample. The team will then design, develop, and deploy a free online digital repository for the categorization of instruments and describe their associated validity evidence.

Aligning the Science Teacher Education Pathway: A Networked Improvement Community

This project will study the activities of a Networked Improvement Community (NIC) as a vehicle to bridge gaps across four identified steps along the science teacher training and development pathways within local contexts of 8 participating universities. The overarching goal of the project is to strengthen the capacity of universities and school districts to reliably produce teachers of science who are knowledgeable about and can effectively enact the Next Generation Science Standards (NGSS), although prepared in varied organizational contexts.

Award Number: 
1908900
Funding Period: 
Mon, 07/01/2019 to Fri, 06/30/2023
Full Description: 

California State University will study the activities of a Networked Improvement Community (NIC) as a vehicle to bridge gaps across four identified steps along the science teacher training and development pathways within local contexts of 8 participating universities (NIC sites). Networked Improvement Community (NIC) will co-create a shared vision and co-defined research agenda between university researchers, science educators and school district practitioners working together to reform teacher education across a variety of local contexts. By studying outcomes of shared supports and teacher tools for use in multiple steps along the science teacher education pathway, researchers will map variation existing in the system and align efforts across the science teacher education pathway. This process will integrate an iterative nature of educational change in local contexts impacting enactment of the NGSS in both university teacher preparation programs and in school district professional training activities and classrooms.

The overarching goal of the project is to strengthen the capacity of universities and school districts to reliably produce teachers of science who are knowledgeable about and can effectively enact the Next Generation Science Standards (NGSS), although prepared in varied organizational contexts. The project will accomplish this goal 1) leveraging the use of an established Networked Improvement Community, composed of science education faculty from eight university campuses and by 2) improving and studying coherence in the steps along the science teacher education pathway within and across these universities and school districts. The project will use a mixed methods approach to data collection and analysis. Consistent with Improvement Science Theory, research questions will be co-defined by all stakeholders.

Developing Organizational Capacity to Improve K-8 Mathematics Teaching and Learning

This project will develop and test a leadership model to improve K-8 mathematics teaching and learning by involving stakeholders across the K-8 spectrum. The project will support teachers, teacher leaders, and administrators in collectively identifying and addressing problems of practice in the teaching and learning of mathematics, and in turn develop plans to improve school and district organizational capacities to support stronger mathematics teaching.

Award Number: 
1907681
Funding Period: 
Mon, 07/01/2019 to Sun, 06/30/2024
Full Description: 

The Developing Organizational Capacity to Improve K-8 Mathematics Teaching and Learning is a 4-year implementation and improvement project. The project will develop and test a leadership model to improve K-8 mathematics teaching and learning by involving stakeholders across the K-8 spectrum. The project will support teachers, teacher leaders, and administrators in collectively identifying and addressing problems of practice in the teaching and learning of mathematics, and in turn develop plans to improve school and district organizational capacities to support stronger mathematics teaching. At the heart of the project is the Elementary Mathematics Leadership (EML) model, which is designed to improve stakeholder understandings of effective math teaching practices. The EML model involves collaboratively identifying classroom-based problems of practice with school and district personnel, designing and implementing professional development aligned with the problems of practice, and iterating those cycles of development, implementation, and revision to assess the model's effectiveness.

The EML model operates at the teacher, school, and district level using a design-based implementation research approach. At the district level, leadership teams in conjunction with researchers will identify problems of practice for which work on those problems will lead to a more coherent mathematics instruction in the district. Following this, professional development and coaching at the teacher level will be designed and implemented to target the problem of practice, with a focus on big ideas within the Common Core State Standards for Mathematics. This phase of the model also includes professional development aimed at school leaders and district administrators to strengthen organizational capacity to support and lead change related to the problem of practice. The final phase of the model calls on researchers, district, and school personnel to engage in an annual redesign of the intervention, making use of data gathered during the school year and evidence about what is happening in classrooms. This design acknowledges the broader policy context in which schools and districts operate as they work towards instructional change. To evaluate the effectiveness of the overall EML model, the project will collect a wide variety of data, including student achievement outcomes using standardized tests; assessments of teachers' mathematical knowledge, instructional practices, and efficacy measures; and measures of leader, administrator, and organizational capacities to support high-quality mathematics instruction. Four districts will be recruited to participate, enacting the model in pairs with a staggered start for one pair of districts to be able to measure treatment effects, using a variation of a switching replications design. Classroom practice and teacher outcomes will be assessed using a variety of MKT assessments, the Mathematical Quality of Instruction (MQI), and the Instructional Quality Assessment (IQA). School level outcomes will be collected via a leadership assessment and interview data, and district level outcomes will be assessed through the use of interview and documentary data. Analysis will include a statistical analysis of the EML model using hierarchical linear modeling, MANOVA/ANOVA, and regression as appropriate at the level of students and teachers, and qualitative analysis and descriptive statistics will be used at the school and district level due to small sample size.

Building Professional Capital in Elementary Science Teaching through a District-wide Networked Improvement Community Model

This project will focus on a networked improvement community (NIC) model of professional learning that shifts K-5 science instruction from traditional approaches to a three-dimensional design as outlined in the Next Generation Science Standards. The project will feature a multi-level model involving university educators and researchers and school district practitioners in an effort to co-defined problems of practice valuable to both parties.

Lead Organization(s): 
Award Number: 
1907471
Funding Period: 
Mon, 07/01/2019 to Thu, 06/30/2022
Full Description: 

This project will focus on a networked improvement community (NIC) model of professional learning that shifts K-5 science instruction from traditional approaches to a three-dimensional design as outlined in the Next Generation Science Standards. The need to make this shift stems from the school district's decision to address inequities in science as some schools offer minimal to no science instruction during the elementary years. The NIC model will draw on expertise from school personnel and university partners to ensure that students will have access to and benefit from authentic model-based inquiry daily in the early grades. This model embraces the challenges of scale and sustainability by targeting the design and substance of professional learning and its organization within the district, balancing integration with existing system infrastructure, and shifting the system based on theory-driven practices. To prepare teachers for this major change, professional development will shift from: (1) training on the use of kit-based curricular materials to professional learning grounded in NGSS-inspired sets of practices and tools; (2) working as individual practitioners to teaching as collaborative investigations; (3) using centralized efforts to distributed knowledge-building and leadership; (4) learning science as decontextualized facts to deep engagement with real-world phenomena; and (5) teaching lessons as prescribed by curriculum to a focus on responsive teaching and building on students' funds of knowledge. The NIC model will provide a pathway for integrating and implementing these shifts via a multilevel, knowledge-building, problem-solving system. This system will go beyond a single focus on improving students' understanding of science content to incorporating teaching practices that advances knowledge about student's written and spoken scientific language and use of explanations and arguments. Through the NIC model all K-5 elementary students in the district will benefit from a rigorous and equitable approach to science learning.

This project will feature a multi-level model involving university educators and researchers and school district practitioners in an effort to co-defined problems of practice valuable to both parties. A mixed methods research design will examine how the NIC model develops professional capital through changes in implementation over multiple iterations. These changes will be captured through short and long-term instruments. Regarding the shorter term, practical measures sensitive to change and directly tied to small manageable, short-term goals will provide quick responses to everyday real-time questions. These measures will help assess specific improvement goals using language relevant and meaningful to researchers and practitioners. For longer term goals, in-depth case studies, interviews, observations, pre-posttests, surveys, and questionnaires will collect data on several variables critical to documenting improvements at the teacher and student levels. Both sets of data will generate knowledge about ambitious and equitable science teaching practices with a focus on students' cultural and linguistic resources and experiences. Through such pathways, knowledge will be generated on teachers' and students' growth as active builders and collaborators in the development of improved learning and experiences. The outcomes will identify critical facets that support advances and sustainability that illuminate variations within the district to better understand what works, for whom, and under what conditions. the research findings will also be used to inform decision making about teaching science at the elementary grades and to further refine equity-based practices, resources, and tools for building on students' funds of knowledge vital to supporting, sustaining, and scaling educational outcomes for all students in the district and beyond.

Understanding the Role of Simulations in K-12 Science and Mathematics Teacher Education

This project will develop and implement a working conference for scholars and practitioners to articulate current use cases and theories of action regarding the use of simulations in PreK-12 science and mathematics teacher education. The conference will be structured to provide opportunities for attendees to share their current research, theoretical models, conceptual views, and use cases focused on the design and use of digital and non-digital simulations for building and assessing K-12 science and mathematics teacher competencies.

Lead Organization(s): 
Award Number: 
1813476
Funding Period: 
Sat, 09/01/2018 to Sat, 08/31/2019
Full Description: 

The recent emergence of updated learning standards in science and mathematics, coupled with increasingly diverse school students across the nation, has highlighted the importance of updating professional learning opportunities for science and mathematics teachers. One promising approach that has emerged is the use of simulations to engage teachers in approximations of practice where the focus is on helping them learn how to engage in ambitious content teaching. In particular, recent technological advances have supported the emergence of new kinds of digital simulations and have brought increased attention to simulations as a tool to enhance teacher learning. This project will develop and implement a working conference for scholars and practitioners to articulate current use cases and theories of action regarding the use of simulations in PreK-12 science and mathematics teacher education. The conference will be structured to provide opportunities for attendees to share their current research, theoretical models, conceptual views, and use cases focused on the design and use of digital and non-digital simulations for building and assessing K-12 science and mathematics teacher competencies.

While the use of simulations in teacher education is neither new nor limited to digital simulation, emerging technological capabilities have enabled digital simulations to become practical in ways not formerly available. The current literature base, however, is dated and the field lacks clear theoretic models or articulated theories of action regarding what teachers could or should learn via simulations, and the essential components of effective learning trajectories. This working conference will be structured to provide opportunities for attending, teacher educators, researchers, professional development facilitators, policy makers, preservice and inservice teachers, and school district leaders to share their current research, theoretical models, conceptual views, and use cases regarding the role of simulations in K-12 science and mathematics teacher education. The conference will be organized around four major goals, including: (1) Define how simulations (digital and non-digital) are conceptualized, operationalized, and utilized in K-12 science and mathematics teacher education; (2) Document and determine the challenges and affordances of the varied contexts, audiences, and purposes for which simulations are used in K-12 science and mathematics teacher education and the variety of investigation methods and research questions employed to investigate the use of simulations in these settings; (3) Make explicit the theories of action and conceptual views undergirding the various simulation models being used in K-12 science and mathematics teacher education; and (4) Determine implications of the current research and development work in this space and establish an agenda for studying the use of simulations in K-12 science and mathematics teacher education. The project will produce a white paper that presents the research and development agenda developed by the working conference, describes a series of use cases describing current and emergent practice, and identifies promising directions for future research and development in this area. Conference outcomes are expected to advance understanding of the varied ways in which digital and non-digital simulations can be used to foster and assess K-12 science and mathematics teacher competencies and initiate a research and development agenda for examining the role of simulations in K-12 science and mathematics teacher education.


Project Videos

2019 STEM for All Video Showcase

Title: Understanding the Role of Simulations in Teacher Preparation

Presenter(s): Lisa Dieker, Angelica Fulchini Scruggs, Heather Howell, Michael Hynes, & Jamie Mikeska


Methods for Assessing Replication

The goal of this project is to formalize subjective ideas about the important concept of replication, provide statistical analyses for evaluating replication studies, provide properties for evaluating the conclusiveness of replication studies, and provide principles for designing conclusive and efficient programs of replication studies.

Lead Organization(s): 
Award Number: 
1841075
Funding Period: 
Sat, 09/01/2018 to Tue, 08/31/2021
Full Description: 

Replication of prior findings and results is a fundamental feature of science and is part of the logic supporting the claim that science is self-correcting. However, there is little prior research on the methodology for studying replication. Research involving meta-analysis and systematic reviews that summarizes a collection of research studies is more common. However, the question of whether the findings from a set of experimental studies replicate one another has received less attention. There is no clearly defined and widely accepted definition of a successful replication study or statistical literature providing methodological guidelines on how to design single replication studies or a set of replication studies. The research proposed here builds this much needed methodology.

The goal of this project is to formalize subjective ideas about the important concept of replication, provide statistical analyses for evaluating replication studies, provide properties for evaluating the conclusiveness of replication studies, and provide principles for designing conclusive and efficient programs of replication studies. It addresses three fundamental problems. The first is how to define replication: What, precisely, should it mean to say that the results in a collection of studies replicate one another? Second, given a definition of replication, what statistical analyses should be done to decide whether the collection of studies replicate one another and what are the properties of these analyses (e.g., sensitivity or statistical power)? Third, how should one or more replication studies be designed to provide conclusive answers to questions of replication? The project has the potential for impact on a range of empirical sciences by providing statistical tools to evaluate the replicability of experimental findings, assessing the conclusiveness of replication attempts, and developing software to help plan programs of replication studies that can provide conclusive evidence of replicability of scientific findings.

Design and Development of Transmedia Narrative-based Curricula to Engage Children in Scientific Thinking and Engineering Design (Collaborative Research: Ellis)

This project will address the need for engineering resources by applying an innovative pedagogy called Imaginative Education (IE) to create middle school engineering curricula. In IE, developmentally appropriate narratives are used to design learning environments that help learners engage with content and organize their knowledge productively. This project will combine IE with transmedia storytelling.

Lead Organization(s): 
Award Number: 
1814033
Funding Period: 
Sun, 07/15/2018 to Thu, 06/30/2022
Full Description: 

Engineering is an important component of the Next Generation Science Standards (NGSS). However, resources for supporting teachers in implementing these standards are scarce. This project will address the need for resources by applying an innovative pedagogy called Imaginative Education (IE) to create middle school engineering curricula. In IE, developmentally appropriate narratives are used to design learning environments that help learners engage with content and organize their knowledge productively. To fully exploit the potential of this pedagogy, this project will combine IE with transmedia storytelling. In transmedia storytelling, different elements of a narrative are spread across a variety of formats (such as books, websites, new articles, videos and other media) in a way that creates a coordinated experience for the user. Once created, the curricula will be implemented in classrooms to research its impact on (1) increasing learners' capacities to engage in both innovative and direct application of engineering concepts, and (2) improving learners' science, technology, engineering, and mathematics (STEM) identity. 

This research will be led by Smith College and Springfield Technical Community College in collaboration with Springfield (MA) Public Schools (SPS). Additional expertise in evaluating the findings will be provided by the Collaborative for Educational Services and an external advisory board of leaders in STEM education and transmedia storytelling. The project will result in the development of a transmedia learning environment that includes two NGSS-aligned, interdisciplinary engineering units and seven lessons that integrate science and engineering. The research study will be implemented in four phases in eight SPS middle schools. Approximately 900 students will participate each year. In Phase 1, the project team will collaborate with SPS teachers to create engineering units, lessons, and standards-based achievement measures. In Phase 2, teachers in the treatment group will participate in professional development (PD) workshops covering IE, transmedia learning environments, structure of the curriculum, and connections to NGSS. In Phase 3 the curricula will be implemented in treatment classrooms and both treatment and control group students will be assessed. In Phase 4, testing and assessment will continue in SPS schools and will be expanded to rural and suburban classrooms. Teachers in these classrooms will use online multimedia PD that will ensure scalability and mirrors the structure and content of in-person PD. Data analysis will provide evidence of whether this imaginative and transmedia educational approach improves students' capacities for using engineering concepts and enhances their STEM identity.


Project Videos

2019 STEM for All Video Showcase

Title: Transforming Engineering Education for Middle School (TEEMS)

Presenter(s): Beth McGinnis-Cavanaugh, Sonia Ellis, & Crystal Ford


Design and Development of Transmedia Narrative-based Curricula to Engage Children in Scientific Thinking and Engineering Design (Collaborative Research: McGinnis-Cavanaugh)

This project will address the need for engineering resources by applying an innovative pedagogy called Imaginative Education (IE) to create middle school engineering curricula. In IE, developmentally appropriate narratives are used to design learning environments that help learners engage with content and organize their knowledge productively. This project will combine IE with transmedia storytelling.

Partner Organization(s): 
Award Number: 
1813572
Funding Period: 
Sun, 07/15/2018 to Thu, 06/30/2022
Project Evaluator: 
Collaborative for Educational Services (CES)
Full Description: 

Engineering is an important component of the Next Generation Science Standards (NGSS). However, resources for supporting teachers in implementing these standards are scarce. This project will address the need for resources by applying an innovative pedagogy called Imaginative Education (IE) to create middle school engineering curricula. In IE, developmentally appropriate narratives are used to design learning environments that help learners engage with content and organize their knowledge productively. To fully exploit the potential of this pedagogy, this project will combine IE with transmedia storytelling. In transmedia storytelling, different elements of a narrative are spread across a variety of formats (such as books, websites, new articles, videos and other media) in a way that creates a coordinated experience for the user. Once created, the curricula will be implemented in classrooms to research its impact on (1) increasing learners' capacities to engage in both innovative and direct application of engineering concepts, and (2) improving learners' science, technology, engineering, and mathematics (STEM) identity. 

This research will be led by Smith College and Springfield Technical Community College in collaboration with Springfield (MA) Public Schools (SPS). Additional expertise in evaluating the findings will be provided by the Collaborative for Educational Services and an external advisory board of leaders in STEM education and transmedia storytelling. The project will result in the development of a transmedia learning environment that includes two NGSS-aligned, interdisciplinary engineering units and seven lessons that integrate science and engineering. The research study will be implemented in four phases in eight SPS middle schools. Approximately 900 students will participate each year. In Phase 1, the project team will collaborate with SPS teachers to create engineering units, lessons, and standards-based achievement measures. In Phase 2, teachers in the treatment group will participate in professional development (PD) workshops covering IE, transmedia learning environments, structure of the curriculum, and connections to NGSS. In Phase 3 the curricula will be implemented in treatment classrooms and both treatment and control group students will be assessed. In Phase 4, testing and assessment will continue in SPS schools and will be expanded to rural and suburban classrooms. Teachers in these classrooms will use online multimedia PD that will ensure scalability and mirrors the structure and content of in-person PD. Data analysis will provide evidence of whether this imaginative and transmedia educational approach improves students' capacities for using engineering concepts and enhances their STEM identity.


Project Videos

2019 STEM for All Video Showcase

Title: Transforming Engineering Education for Middle School (TEEMS)

Presenter(s): Beth McGinnis-Cavanaugh, Sonia Ellis, & Crystal Ford


Science Communities of Practice Partnership

This project will study implementation of an effective professional learning model for elementary science teachers that includes teacher leaders, administrators and university educators in a system perspective for improving science instruction in ways that make it sustainable.

Award Number: 
1813012
Funding Period: 
Wed, 08/01/2018 to Sun, 07/31/2022
Full Description: 

This project will study implementation of an effective professional learning model for elementary science teachers that includes teacher leaders, administrators and university educators in a system perspective for improving science instruction in ways that make it sustainable. The working model involves reciprocal communities of practice, which are groups of teachers, leaders and administrators that focus on practical tasks and how to achieve them across these stakeholder perspectives. The project will provide evidence about the specific components of the professional development model that support sustainable improvement in science teaching, will test the ways that teacher ownership and organizational conditions mediate instructional change, and will develop four tools for facilitating the teacher learning and the accompanying capacity building. In this way, the project will produce practical knowledge and tools necessary for other school districts nationwide to create professional learning that is tailored to their contexts and therefore sustainable.

This study posits that communication among district teachers, teacher leaders, and administrators, and a sense of ownership for improved instruction among teachers can support sustainable change. As such, it tests a model that fosters communication and ownership through three reciprocal communities of practice--one about district leadership including one teacher per school, coaches and university faculty; another about lesson study including teachers, coaches and faculty; and a third about instructional innovation including teachers and administrators, facilitated by coaches. The research design seeks to inform what the communities of practice add to the effects in a quasi-experimental study involving 72 third to fifth grade teachers and 6500 students in four urban school districts. Mixed methodologies will be used to examine shifts in science teaching over three years, testing the professional development model and the mediating roles of reform ownership and organizational conditions.

Critical Issues in Mathematics Education 2018

This conference will continue the workshop series Critical Issues in Mathematics Education (CIME). The topic for CIME 2018 will be "Access to mathematics by opening doors for students currently excluded from mathematics". The CIME workshops engage professional mathematicians, education researchers, teachers, and policy makers in discussions of issues critical to the improvement of mathematics education from the elementary grades through undergraduate years.

Award Number: 
1827412
Funding Period: 
Thu, 03/01/2018 to Thu, 02/28/2019
Full Description: 

This conference will continue the workshop series, Critical Issues in Mathematics Education (CIME) on teaching and learning mathematics, initiated by the Mathematical Sciences Research Institute (MSRI) in 2004. The topic for CIME 2018 will be "Access to mathematics by opening doors for students currently excluded from mathematics". The CIME workshops engage professional mathematicians, education researchers, teachers, and policy makers in discussions of issues critical to the improvement of mathematics education from the elementary grades through undergraduate years. Sessions will share relevant programmatic efforts and innovative research that have been shown to maintain or increase students' engagement and interests in mathematics across K-12, undergraduate and graduate education. The sessions will focus particularly on reproducible efforts that affirm those students' identities and their diverse intellectual resources and lived experience.

The CIME workshops impact three distinct communities: research mathematicians, mathematics educators (K-16), and education researchers. Participants learn about research and development efforts that can enhance their own work and the contributions they can make to solving issues in mathematics education. Participants also connect with others concerned about those issues. This workshop will also focus on developing action plans that participants can implement once they return to their institutions. There is also a focus on recruitment of leaders of mathematics departments, teachers, and other leaders in mathematics education across K-12, undergraduate education and graduate education in order to examine systemic changes that can be made to increase access, engagement, and interest in mathematics.

Pages

Subscribe to Post-secondary Faculty