Quasi-experimental

Systemic Formative Assessment to Promote Mathematics Learning in Urban Elementary Schools

This project builds on the study of the Ongoing Assessment Project's (OGAP) math assessment intervention on elementary teachers and students and combines the intervention with research-based understandings of systemic reform. This project will produce concrete tools, routines, and practices that can be applied to strengthen programs' implementation by ensuring the strategic support of school and district leaders.

Lead Organization(s): 
Award Number: 
1621333
Funding Period: 
Thu, 09/15/2016 to Sat, 02/29/2020
Full Description: 

Districts have long struggled to implement instructional programming in ways that meaningfully and sustainably impact teaching and learning. Systemic education reform is based on the hypothesis that prevailing patterns of incoherence and misalignment in an educational system can send mixed messages to local implementers as they try to respond to various cues and incentives in the environment. Systemic reform seeks to bring alignment to education systems in multiple ways, including consistency across instructional philosophies, alignment across grade levels, and vertical coherence from district to schools to classrooms. This project builds on the Consortium for Policy Research in Education's (CPRE) ongoing, NSF-funded experimental study of the impacts of the Ongoing Assessment Project's (OGAP) math assessment intervention on elementary teachers and students in Philadelphia-area schools. The project will combine the OGAP math intervention with research-based understandings of systemic reform. OGAP is based upon established theory and research demonstrating the impact of teachers' use of ongoing short- and medium-cycle formative assessment on student learning. It combines these understandings with recent research on learning trajectories within mathematics content domains. By bringing to bear the strengths of all three of these areas of research - formative assessment, learning trajectories, and systemic reform - the project promises a significant contribution to the knowledge base about the application of math learning research to classroom instruction on a large scale. This project will produce concrete tools, routines, and practices that can be applied to strengthen programs' implementation by ensuring the strategic support of school and district leaders. This project is funded by the Discovery Research PreK-12 (DRK-12) and EHR Core Research (ECR) Programs. The DRK-12 program supports research and development on STEM education innovations and approaches to teaching, learning, and assessment. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field.

CPRE and the School District of Philadelphia (SDP) will establish a research-practice partnership focused on developing, implementing, refining, and testing a systemic support model to strengthen implementation of the OGAP math intervention in elementary schools. CPRE's current experimental study of OGAP's impacts reveals, preliminarily, statistically significant positive effects on teacher knowledge and student learning. As a result, SDP has decided to expand OGAP into an additional 60 schools in 2016-17. However, the current OGAP study has also revealed weak implementation stemming from a lack of consistent leadership support for the intervention. The project will address this implementation challenge by developing, refining, supporting, and documenting a systemic support component that will accompany OGAP's classroom-level implementation. The systemic supports will be developed by a research-practice partnership between CPRE; SDP; OGAP; the Graduate School of Education at the University of Pennsylvania (PennGSE); and the Philadelphia Education Research Consortium (PERC). The team will use principles of design-based implementation research to iteratively refine and improve the systemic support model. Along with the design and development of the systemic support model, the project will conduct a mixed-methods study of its impacts and roll-out. A three-armed quasi-experimental study will examine the differential impacts of OGAP, with and without systemic supports, and business-as-usual math programming on teacher and student outcomes. A mixed-methods study will examine teacher and administrator experiences in both treatment groups, and will provide feedback to inform the iterative development of the systemic support model.

InquirySpace 2: Broadening Access to Integrated Science Practices

This project will create technology-enhanced classroom activities and resources that increase student learning of science practices in high school biology, chemistry, and physics. InquirySpace will incorporate several innovative technological and pedagogical features that will enable students to undertake scientific experimentation that closely mirrors current science research and learn what it means to be a scientist.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1621301
Funding Period: 
Thu, 09/15/2016 to Tue, 08/31/2021
Full Description: 

This project will create technology-enhanced classroom activities and resources that increase student learning of science practices in high school biology, chemistry, and physics courses. The project addresses the urgent national priority to improve science education as envisioned in the Next Generation Science Standards (NGSS) by focusing less on learning facts and equations and instead providing students with the time, skills, and resources to experience the conduct of science and what it means to be a scientist. This project builds on prior work that created a sequence of physics activities that significantly improved students' abilities to undertake data-based experiments and led to productive independent investigations. The goal of the InquirySpace project is to improve this physics sequence, extend the approach to biology and chemistry, and adapt the materials to the needs of diverse students by integrating tailored formative feedback in real time. The result will be student and teacher materials that any school can use to allow students to experience the excitement and essence of scientific investigations as an integral part of science instruction. The project plans to create and iteratively revise learning materials and technologies, and will be tested in 48 diverse classroom settings. The educational impact of the project's approach will be compared with that of business-as-usual approaches used by teachers to investigate to what extent it empowers students to undertake self-directed experiments. To facilitate the widest possible use of the project, a complete set of materials, software, teacher professional development resources, and curriculum design documents will be available online at the project website, an online teacher professional development course, and teacher community sites. The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

InquirySpace will incorporate several innovative technological and pedagogical features that will enable students to undertake scientific experimentation that closely mirrors current science research. These features will include (1) educational games to teach data analysis and interpretation skills needed in the approach, (2) reduced dependence on reading and writing through the use of screencast instructions and reports, (3) increased reliance on graphical analysis that can make equations unnecessary, and (4) extensive use of formative feedback generated from student logs. The project uses an overarching framework called Parameter Space Reasoning (PSR) to scaffold students through a type of experimentation applicable to a very large class of experiments. PSR involves an integrated set of science practices related to a question that can be answered with a series of data collection runs for different values of independent variables. Data can be collected from sensors attached to the computer, analysis of videos, scientific databases, or computational models. A variety of visual analytic tools will be provided to reveal patterns in the graphs. Research will be conducted in three phases: design and development of technology-enhanced learning materials through design-based research, estimation of educational impact using a quasi-experimental design, and feasibility testing across diverse classroom settings. The project will use two analytical algorithms to diagnose students' learning of data analysis and interpretation practices so that teachers and students can modify their actions based on formative feedback in real time. These algorithms use computationally optimized calculations to model the growth of student thinking and investigation patterns and provide actionable information to teachers and students almost instantly. Because formative feedback can improve instruction in any field, this is a major development that has wide potential.

Learning Evolution through Human and Non-Human Case Studies

This project will develop and test two curriculum units on the topic of evolution for high school general biology courses, with one unit focusing primarily on human case studies to teach evolution and one unit focusing primarily on case studies of evolution in other species. The two units will be compared to examine how different approaches to teaching evolution affect students and teachers.

Lead Organization(s): 
Award Number: 
1621194
Funding Period: 
Thu, 09/15/2016 to Tue, 08/31/2021
Full Description: 

This project aligns with Alabama's College & Career-Ready Standards (CCRS) for biology in grades 9-12 relating to Unity and Diversity. These standards are based on the Next Generation Science Standards (NGSS) and go into effect during the 2016-2017 school year. Building on prior work (DRL-119468), this project will develop and test two curriculum units on the topic of evolution for high school general biology courses, with one unit focusing primarily on human case studies to teach evolution and one unit focusing primarily on case studies of evolution in other species. The two units will be compared to examine how different approaches to teaching evolution affect students and teachers. The project will also develop and field test a Cultural and Religious Sensitivity (CRS) Resource to provide teachers with strategies for creating supportive learning environments where understanding of the scientific account of evolution is aided while also acknowledging the cultural controversy associated with learning about evolution. The impacts on student and teacher outcomes of using the curriculum units and the CRS Resource will be tested in classrooms by comparing the outcomes of the human versus non-human units, and by using or not using classroom strategies from the CRS Resource.

The project will examine student and teacher outcomes of four treatment groups: 1) Curriculum Unit 1, 2) Curriculum Unit 1 with the CRS Resource, 3) Curriculum Unit 2, and 4) Curriculum Unit 2 with the CRS Resource. The research questions are: 1) In what ways does using examples of human versus non-human evolution to teach core evolutionary concepts affect understanding of, acceptance of, and motivation to learn about evolution among high school introductory biology students? 2) In what ways do using teaching strategies that focus on acknowledging the cultural controversy about evolution using a procedural neutrality approach affect high school introductory biology teachers' comfort and confidence with teaching evolution? 3) In what ways does using examples of human versus non-human evolution to teach fundamental evolutionary concepts in conjunction with teaching strategies that focus on acknowledging the cultural controversy about evolution using a procedural neutrality approach affect understanding of, acceptance of, and motivation to learn about evolution among high school introductory biology students? And 4) In what ways does using examples of human versus non-human evolution to teach fundamental evolutionary concepts in conjunction with teaching strategies that focus on acknowledging the cultural controversy about evolution using a procedural neutrality approach affect high school introductory biology teachers' comfort and confidence with teaching evolution? The project will use a 2 X 2 X 2 mixed factorial quasi-experimental research design to answer these questions, and will include a total of 32 teachers, 8 in each treatment group, along with approximately 800 students. Each assessment will be administered as a pretest two weeks prior to starting the curriculum unit and as a posttest immediately after completing the unit. Test scores will be the within-subjects factors, and Curriculum Unit and CRS Resource will be the between-subjects factors.

Geological Models for Explorations of Dynamic Earth (GEODE): Integrating the Power of Geodynamic Models in Middle School Earth Science Curriculum

This project will develop and research the transformational potential of geodynamic models embedded in learning progression-informed online curricula modules for middle school teaching and learning of Earth science. The primary goal of the project is to conduct design-based research to study the development of model-based curriculum modules, assessment instruments, and professional development materials for supporting student learning of (1) plate tectonics and related Earth processes, (2) modeling practices, and (3) uncertainty-infused argumentation practices.

Lead Organization(s): 
Award Number: 
1621176
Funding Period: 
Mon, 08/15/2016 to Fri, 07/31/2020
Full Description: 

This project will contribute to the Earth science education community's understanding of how engaging students with dynamic computer-based systems models supports their learning of complex Earth science concepts regarding Earth's surface phenomena and sub-surface processes. It will also extend the field's understandings of how students develop modeling practices and how models are used to support scientific endeavors. This research will shed light on the role uncertainty plays when students use models to develop scientific arguments with model-based evidence. The GEODE project will directly involve over 4,000 students and 22 teachers from diverse school systems serving students from families with a variety of socioeconomic, cultural, and racial backgrounds. These students will engage with important geoscience concepts that underlie some of the most critical socio-scientific challenges facing humanity at this time. The GEODE project research will also seek to understand how teachers' practices need to change in order to take advantage of these sophisticated geodynamic modeling tools. The materials generated through design and development will be made available for free to all future learners, teachers, and researchers beyond the participants outlined in the project.

The GEODE project will develop and research the transformational potential of geodynamic models embedded in learning progression-informed online curricula modules for middle school teaching and learning of Earth science. The primary goal of the project is to conduct design-based research to study the development of model-based curriculum modules, assessment instruments, and professional development materials for supporting student learning of (1) plate tectonics and related Earth processes, (2) modeling practices, and (3) uncertainty-infused argumentation practices. The GEODE software will permit students to "program" a series of geologic events into the model, gather evidence from the emergent phenomena that result from the model, revise the model, and use their models to explain the dynamic mechanisms related to plate motion and associated geologic phenomena such as sedimentation, volcanic eruptions, earthquakes, and deformation of strata. The project will also study the types of teacher practices necessary for supporting the use of dynamic computer models of complex phenomena and the use of curriculum that include an explicit focus on uncertainty-infused argumentation.

Supporting Success in Algebra: A Study of the Implementation of Transition to Algebra

The project will research the implementation of Transition to Algebra, a year-long mathematics course for underprepared ninth grade students taken concurrently with Algebra 1 to provide additional support, and its impact on students' attitudes and achievement in mathematics in combination with teachers' instruction and the types of supports teachers need to successfully implement the intervention.

Partner Organization(s): 
Award Number: 
1621011
Funding Period: 
Sat, 10/01/2016 to Wed, 09/30/2020
Full Description: 

The project will research the impact and implementation of Transition to Algebra, a year-long mathematics course for underprepared ninth grade students taken concurrently with Algebra 1 to provide additional support. Nationally, there is a need for programs that support students' learning of algebra and that provide research-based resources and models particularly for students in need of additional support. The design of the Transition to Algebra curriculum reflects the idea that students underprepared for Algebra 1 can benefit from very specific help in building the logic of algebra by connecting arithmetic pattern and algebraic structure. The materials feature the use of mental mathematics, puzzles, explorations, and student dialogues to connect arithmetic pattern to algebraic structure. These features should encourage students to expect mathematics ideas to make sense, and to build algebraic habits of mind and problem solving stamina. The research will investigate the effects of the curriculum on students' algebra achievement and their attitudes towards mathematics. The Discovery Research PreK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

The research questions examine the impact of the Transition to Algebra course on students' attitudes and achievement in mathematics in combination with teachers' instruction and the types of supports teachers need to successfully implement the intervention. The project will use a pre-post quasi-experimental design, along with propensity score methods to reduce selection bias threats, to examine the implementation in approximately 35 treatment schools and 35 comparison schools. Qualitative and quantitative data will be collected and analyzed to address research questions. The study will also investigate how teachers use and adapt Transition to Algebra materials, and the supports critical to successful implementation. For example, the study will examine whether and how school and district activities such as common planning time, coaching, and other professional development experiences influence the implementation fidelity of the curriculum. Qualitative data will be collected through interviews and classroom observations. Quantitative data will be collected using student and teacher surveys, an algebra readiness assessment, a standardized end-of-course assessment, and students' scores on state tests.

Modest Supports for Sustaining Professional Development Outcomes over the Long-Term

This study will investigate factors influencing the persistence of teacher change after professional development (PD) experiences, and will examine the extent to which modest supports for science teaching in grades K-5 sustain PD outcomes over the long term.

Lead Organization(s): 
Award Number: 
1620979
Funding Period: 
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description: 

This study will investigate factors influencing the persistence of teacher change after professional development (PD) experiences, and will examine the extent to which modest supports for science teaching in grades K-5 sustain PD outcomes over the long term. Fifty K-12 teachers who completed one of four PD programs situated in small, rural school districts will be recruited for the study, and they will participate in summer refresher sessions for two days, cluster meetings at local schools twice during the academic year, and optional Webinar sessions two times per year. Electronic supports for participants will include a dedicated email address, a project Facebook page, a biweekly newsletter, and access to archived Webinars on a range of topics related to teaching elementary school science. Modest support for replacement of consumable supplies needed for hands-on classroom activities will also be provided. The project will examine the extent to which these modest supports individually and collectively foster the sustainability of PD outcomes in terms of the instructional time devoted to science, teacher self-efficacy in science, and teacher use of inquiry-based instructional strategies. The effects of contextual factors on sustainability of PD outcomes will also be examined.

This longitudinal study will seek answers to three research questions: 1) To what extent do modest supports foster the sustainability of professional development outcomes in: a) instructional time in science; b) teachers' self-efficacy in science; and c) teachers' use of inquiry-based instructional strategies? 2) Which supports are: a) the most critical for sustainability of outcomes; and b) the most cost-effective; and 3) What contextual factors support or impede the sustainability of professional development outcomes? The project will employ a mixed-methods research design to examine the effects of PD in science among elementary schoolteachers over a 10 to 12 year period that includes a 3-year PD program, a 4-6 year span after the initial PD program, and a 3-year intervention of modest supports. Quantitative and qualitative data will be collected from multiple sources, including: a general survey of participating teachers regarding their beliefs about science, their instructional practices, and their instructional time in science; a teacher self-efficacy measure; intervention feedback surveys; electronic data sources associated with Webinars; teacher interviews; school administrator interviews; and receipts for purchases of classroom supplies. Quantitative data from the teacher survey and self-efficacy measure will be analyzed using hierarchical modeling to examine growth rates after the original PD and the change in growth after the provision of modest supports. Data gathered from other sources will be tracked, coded, and analyzed for each teacher, and linked to the survey and self-efficacy data for analysis by individual teacher, by grade level, by school, by district, and by original PD experience. Together, these data will enable the project team to address the project's research questions, with particular emphasis on determining the extent to which teachers make use of the various supports offered, and identifying the most cost-effective and critical supports.

An Online STEM Career Exploration and Readiness Environment for Opportunity Youth

This project aims to create a web-based STEM Career Exploration and Readiness Environment (CEE-STEM) that will support opportunities for youth ages 16-24 who are neither in school nor are working, in rebuilding engagement in STEM learning and developing STEM skills and capacities relevant to diverse postsecondary education/training and employment pathways.

Award Number: 
1620904
Funding Period: 
Thu, 09/15/2016 to Mon, 08/31/2020
Full Description: 

CAST, the University of Massachusetts-Amherst, and YouthBuild USA aim to create a web-based STEM Career Exploration and Readiness Environment (CEE-STEM). This will support opportunities for youth ages 16-24 who are neither in school nor are working, in rebuilding engagement in STEM learning and developing STEM skills and capacities relevant to diverse postsecondary education/training and employment pathways. The program will provide opportunity youth with a personalized and portable tool to explore STEM careers, demonstrate their STEM learning, reflect on STEM career interests, and take actions to move ahead with STEM career pathways of interest.

The proposed program addresses two critical and interrelated aspects of STEM learning for opportunity youth: the development of STEM foundational knowledge; and STEM engagement, readiness and career pathways. These aspects of STEM learning are addressed through an integrated program model that includes classroom STEM instruction; hands-on job training in career pathways including green construction, health care, and technology.


Project Videos

2020 STEM for All Video Showcase

Title: STEMfolio: A Portfolio Builder & Career Exploration Tool

Presenter(s): Tracey Hall

2019 STEM for All Video Showcase

Title: Building a Diverse STEM Talent Pipeline: Finding What Works

Presenter(s): Tracey Hall

2018 STEM for All Video Showcase

Title: Bridging the Gap Between Ability and Opportunity in STEM

Presenter(s): Sam Johnston


CAREER: Multilevel Mediation Models to Study the Impact of Teacher Development on Student Achievement in Mathematics

This project will develop a comprehensive framework to inform and guide the analytic design of teacher professional development studies in mathematics. An essential goal of the research is to advance a science of teaching and learning in ways that traverse both research and education.

Lead Organization(s): 
Award Number: 
1552535
Funding Period: 
Thu, 09/01/2016 to Tue, 08/31/2021
Full Description: 

This is a Faculty Early Career Development Program (CAREER) project. The CAREER program is a National Science Foundation-wide activity that offers the most prestigious awards in support of junior faculty who exemplify the role of teacher-scholars through outstanding research, excellent education and the integration of education and research. The intellectual merit and broader impacts of this study lie in two complementary contributions of the project. First, the development of the statistical framework for the design of multilevel mediation studies has significant potential for broad impact because it develops a core platform that is transferable to other STEM (science, technology, engineering, and mathematics) education areas and STEM disciplines. Second, the development of software and curricular materials to implement this framework further capitalize on the promise of this work because it distributes the results in an accessible manner to diverse sets of research and practitioner groups across STEM education areas and STEM disciplines. Together, the components of this project will substantially expand the scope and quality of evidence generated through mathematics professional development and, more generally, multilevel mediation studies throughout STEM areas by increasing researchers' capacity to design valid and comprehensive studies of the theories of action and change that underlie research programs.

This project will develop a comprehensive framework to inform and guide the analytic design of teacher professional development studies in mathematics. The proposed framework incorporates four integrated research and education components: (1) develop statistical formulas and tools to guide the optimal design of experimental and non-experimental multilevel mediation studies in the presence of measurement error, (2) develop empirical estimates of the parameters needed to implement these formulas to design teacher development studies in mathematics, (3) develop free and accessible software to execute this framework, and (4) develop training materials and conduct workshops on the framework to improve the capacity of the field to design effective and efficient studies of teacher development. An essential goal of the research is to advance a science of teaching and learning in ways that traverse both research and education.

Science, Technology, Engineering and Mathematics Scholars Teacher Academy Resident System

This project will investigate the effectiveness of a teacher academy resident model to recruit, license, induct, employ, and retain middle school and secondary teachers for high-need schools in the South. It will prepare new, highly-qualified science and mathematics teachers from historically Black universities in high-needs urban and rural schools with the goal of increasing teacher retention and diversity rates.

Lead Organization(s): 
Award Number: 
1621325
Funding Period: 
Fri, 07/15/2016 to Wed, 06/30/2021
Full Description: 

This project at Jackson State University will investigate the effectiveness of a teacher academy resident model to recruit, license, induct, employ, and retain middle school and secondary science and mathematics teachers for high-need schools in the South. It will prepare new, highly-qualified science and mathematics teachers from historically Black universities in high-needs urban and rural schools. The project involves a partnership among three historically Black universities (Jackson, State University, Xavier University of Louisiana, and the University of Arkansas at Pine Bluff), and diverse urban and rural school districts in Jackson, Mississippi; New Orleans, Louisiana; and Pine Bluff Arkansas region that serve more than 175,000 students.

Participants will include 150 middle and secondary school teacher residents who will gain clinical mentored experience and develop familiarity with local schools. The 150 teacher residents supported by the program to National Board certification will obtain: state licensure/certification in science teaching, a master's degree, and initiation. The goal is to increase teacher retention and diversity rates. The research question guiding this focus is: Will training STEM graduates have a significant effect on the quality of K-12 instruction, teacher efficacy and satisfaction, STEM teacher retention, and students? Science and mathematics achievement? A quasi-experimental design will be used to evaluate project's effectiveness.

CAREER: Making Science Visible: Using Visualization Technology to Support Linguistically Diverse Middle School Students' Learning in Physical and Life Sciences

Award Number: 
1552114
Funding Period: 
Wed, 06/01/2016 to Mon, 05/31/2021
Full Description: 

The growing diversity in public schools requires science educators to address the specific needs of English language learners (ELLs), students who speak a language other than English at home. Although ELLs are the fastest-growing demographic group in classrooms, many are historically underserved in mainstream science classrooms, particularly those from underrepresented minority groups. The significant increase of ELLs at public schools poses a challenge to science teachers in linguistically diverse classrooms as they try to support and engage all students in learning science. The proposed project will respond to this urgent need by investigating the potential benefits of interactive, dynamic visualization technologies, including simulations, animations, and visual models, in supporting science learning for all middle school students, including ELLs. This project will also identify design principles for developing such technology, develop additional ways to support student learning, and provide new guidelines for effective science teachers' professional development that can assist them to better serve students from diverse language backgrounds. The project has the potential to transform traditional science instruction for all students, including underserved ELLs, and to broaden their participation in science.

In collaboration with eighth grade science teachers from two low-income middle schools in North Carolina, the project will focus on three objectives: (1) develop, test, and refine four open-source, web-based inquiry units featuring dynamic visualizations on energy and matter concepts in physical and life sciences, aligned with the Next Generation Science Standards (NGSS); (2) investigate how dynamic visualizations can engage eighth-grade ELLs and native-English-speaking students in science practices and improve their understanding of energy and matter concepts; and (3) investigate which scaffolding approaches can help maximize ELLs' learning with visualizations. Research questions include: (1) Which kinds of dynamic visualizations (simulations, animations, visual models) lead to the best learning outcomes for all students within the four instructional science units?; (2) Do ELLs benefit more from visualizations (or particular kinds of visualizations) than do native-English-speaking students?; and (3) What kinds of additional scaffolding activities (e.g., critiquing arguments vs. generating arguments) are needed by ELLs in order to achieve the greatest benefit? The project will use design-based research and mixed-methods approaches to accomplish its research objectives and address these questions. Furthermore, it will help science teachers develop effective strategies to support students' learning with visualizations. Products from this project, including four NGSS-aligned web-based inquiry units, the visualizations created for the project, professional development materials, and scaffolding approaches for teachers to use with ELLs, will be freely available through a project website and multiple professional development networks. The PI will collaborate with an advisory board of experts to develop the four instructional units, visualizations, and scaffolds, as well as with the participating teachers to refine these materials in an iterative fashion. Evaluation of the materials and workshops will be provided each year by the advisory board members, and their feedback will be used to improve design and implementation for the next year. The advisory board will also provide summative evaluation of student learning outcomes and will assess the success of the teachers' professional development workshops.

Pages

Subscribe to Quasi-experimental