Effectiveness

GRIDS: Graphing Research on Inquiry with Data in Science

The Graphing Research on Inquiry with Data in Science (GRIDS) project will investigate strategies to improve middle school students' science learning by focusing on student ability to interpret and use graphs. GRIDS will undertake a comprehensive program to address the need for improved graph comprehension. The project will create, study, and disseminate technology-based assessments, technologies that aid graph interpretation, instructional designs, professional development, and learning materials.

Award Number: 
1418423
Funding Period: 
Mon, 09/01/2014 to Sat, 08/31/2019
Full Description: 

The Graphing Research on Inquiry with Data in Science (GRIDS) project is a four-year full design and development proposal, addressing the learning strand, submitted to the DR K-12 program at the NSF. GRIDS will investigate strategies to improve middle school students' science learning by focusing on student ability to interpret and use graphs. In middle school math, students typically graph only linear functions and rarely encounter features used in science, such as units, scientific notation, non-integer values, noise, cycles, and exponentials. Science teachers rarely teach about the graph features needed in science, so students are left to learn science without recourse to what is inarguably a key tool in learning and doing science. GRIDS will undertake a comprehensive program to address the need for improved graph comprehension. The project will create, study, and disseminate technology-based assessments, technologies that aid graph interpretation, instructional designs, professional development, and learning materials.

GRIDS will start by developing the GRIDS Graphing Inventory (GGI), an online, research-based measure of graphing skills that are relevant to middle school science. The project will address gaps revealed by the GGI by designing instructional activities that feature powerful digital technologies including automated guidance based on analysis of student generated graphs and student writing about graphs. These materials will be tested in classroom comparison studies using the GGI to assess both annual and longitudinal progress. Approximately 30 teachers selected from 10 public middle schools will participate in the project, along with approximately 4,000 students in their classrooms. A series of design studies will be conducted to create and test ten units of study and associated assessments, and a minimum of 30 comparison studies will be conducted to optimize instructional strategies. The comparison studies will include a minimum of 5 experiments per term, each with 6 teachers and their 600-800 students. The project will develop supports for teachers to guide students to use graphs and science knowledge to deepen understanding, and to develop agency and identity as science learners.

Developing and Testing the Internship-inator, a Virtual Internship in STEM Authorware System

The Internship-inator is an authorware system for developing and testing virtual internships in multiple STEM disciplines. In a virtual internship, students are presented with a complex, real-world STEM problem for which there is no optimal solution. Students work in project teams to read and analyze research reports, design and perform experiments using virtual tools, respond to the requirements of stakeholders and clients, write reports and present and justify their proposed solutions. 

Award Number: 
1418288
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

Ensuring that students have the opportunities to experience STEM as it is conducted by scientists, mathematicians and engineers is a complex task within the current school context. This project will expand access for middle and high school students to virtual internships, by enabling STEM content developers to design and customize virtual internships. The Internship-inator is an authorware system for developing and testing virtual internships in multiple STEM disciplines. In a virtual internship, students are presented with a complex, real-world STEM problem for which there is no optimal solution. Students work in project teams to read and analyze research reports, design and perform experiments using virtual tools, respond to the requirements of stakeholders and clients, write reports and present and justify their proposed solutions. The researchers in this project will work with a core development network to develop and refine the authorware, constructing up to a hundred new virtual internships and a user group of more than 70 STEM content developers. The researchers will iteratively analyze the performance of the authorware, focusing on optimizing the utility and the feasibility of the system to support virtual internship development. They will also examine the ways in which the virtual internships are implemented in the classroom to determine the quality of the STEM internship design and influence on student learning.

The Intership-inator builds on over ten years of NSF support for the development of Syntern, a platform for deploying virtual internships that has been used in middle schools, high schools, informal science programs, and undergraduate education. In the current project, the researchers will recruit two waves of STEM content developers to expand their current core development network. A design research perspective will be used to examine the ways in which the developers interact with the components of the authorware and to document the influence of the virtual internships on student learning. The researchers will use a quantitative ethnographic approach to integrate qualitative data from surveys and interviews with the developers with their quantitative interactions with the authorware and with student use and products from pilot and field tests of the virtual internships. Data-mining and learning analytics will be used in combination with hierarchical linear modeling, regression techniques and propensity score matching to structure the quasi-experimental research design. The authorware and the multiple virtual internships will provide researchers, developers, and teachers a rich learning environment in which to explore and support students' learning of important college and career readiness content and disciplinary practices. The findings of the use of the authorware will inform STEM education about the important design characteristics for authorware that supports the work of STEM content and curriculum developers.

Driven to Discover: Citizen Science Inspires Classroom Investigation

This project utilizes existing citizen science programs as springboards for professional development for teachers during an intensive summer workshop. The project curriculum helps teachers use student participation in citizen science to engage them in the full complement of science practices; from asking questions, to conducting independent research, to sharing findings.

Award Number: 
1417777
Funding Period: 
Wed, 10/01/2014 to Sun, 09/30/2018
Full Description: 

Citizen science refers to partnerships between volunteers and scientists that answer real world questions. The target audiences in this project are middle and high school teachers and their students in a broad range of settings: two urban districts, an inner-ring suburb, and three rural districts. The project utilizes existing citizen science programs as springboards for professional development for teachers during an intensive summer workshop. The project curriculum helps teachers use student participation in citizen science to engage them in the full complement of science practices; from asking questions, to conducting independent research, to sharing findings. Through district professional learning communities (PLCs), teachers work with district and project staff to support and demonstrate project implementation. As students and their teachers engage in project activities, the project team is addressing two key research questions: 1) What is the nature of instructional practices that promote student engagement in the process of science?, and 2) How does this engagement influence student learning, with special attention to the benefits of engaging in research presentations in public, high profile venues? Key contributions of the project are stronger connections between a) ecology-based citizen science programs, STEM curriculum, and students' lives and b) science learning and disciplinary literacy in reading, writing and math.

Research design and analysis are focused on understanding how professional development that involves citizen science and independent investigations influences teachers' classroom practices and student learning. The research utilizes existing instruments to investigate teachers' classroom practices, and student engagement and cognitive activity: the Collaboratives for Excellence in Teacher Preparation and Classroom Observation Protocol, and Inquiring into Science Instruction Observation Protocol. These instruments are used in classroom observations of a stratified sample of classes whose students represent the diversity of the participating districts. Curriculum resources for each citizen science topic, cross-referenced to disciplinary content and practices of the NGSS, include 1) a bibliography (books, web links, relevant research articles); 2) lesson plans and student science journals addressing relevant science content and background on the project; and 3) short videos that help teachers introduce the projects and anchor a digital library to facilitate dissemination. Impacts beyond both the timeframe of the project and the approximately 160 teachers who will participate are supported by curriculum units that address NGSS life science topics, and wide dissemination of these materials in a variety of venues. The evaluation focuses on outcomes of and satisfaction with the summer workshop, classroom incorporation, PLCs, and student learning. It provides formative and summative findings based on qualitative and quantitative instruments, which, like those used for the research, have well-documented reliability and validity. These include the Science Teaching Efficacy Belief Instrument to assess teacher beliefs; the Reformed Teaching Observation Protocol to assess teacher practices; the Standards Assessment Inventory to assess PLC quality; and the Scientific Attitude Inventory to assess student attitudes towards science. Project deliverables include 1) curriculum resources that will support engagement in five existing citizen science projects that incorporate standards-based science content; 2) venues for student research presentations that can be duplicated in other settings; and 3) a compilation of teacher-adapted primary scientific research articles that will provide a model for promoting disciplinary literacy. The project engages 40 teachers per year and their students.

Computer Science in Secondary Schools (CS3): Studying Context, Enactment, and Impact

This project will examine the relationships among the factors that influence the implementation of the Exploring Computer Science (ECS), a pre-Advanced Placement curriculum that prepares students for further study in computer science. This study elucidates how variation in curricular implementation influences student learning and determines not only what works, but also for whom and under what circumstances.

Lead Organization(s): 
Award Number: 
1418149
Funding Period: 
Fri, 08/01/2014 to Tue, 07/31/2018
Full Description: 

Computational thinking is an important set of 21st century knowledge and skills that has implications for the heavily technological world in which we live. Multiple industries indicate the under supply of those trained to be effective in the computer science workforce. In addition, there are increasing demands for broadening the participation in the computer science workforce by women and members of minority populations. SRI International will examine the relationships among the factors that influence the implementation of the Exploring Computer Science (ECS), a pre-Advanced Placement curriculum that prepares students for further study in computer science. SRI will work in partnership with the ECS curriculum developers, teachers, and the nonprofit Code.org who are involved in the scaling of ECS. This study elucidates how variation in curricular implementation influences student learning and determines not only what works, but also for whom and under what circumstances.

SRI will conduct a pilot study in which they develop, pilot, and refine measures as they recruit school districts for the implementation study. The subsequent implementation study will be a 2 year examination of curriculum enactment, teacher practice, and evidence of student learning. Because no comparable curriculum currently exists, the study will examine the conditions needed to implement the ECS curriculum in ways that improve student computational thinking outcomes rather than determine whether the ECS curriculum is more effective than other CS-related curricula. The study will conduct two kinds of analyses: 1) an analysis of the influence of ECS on student learning gains, and 2) an analysis of the relationship between classroom-level implementation and student learning gains. Because of the clustered nature of the data (students nested within classrooms nested within schools), the project will use hierarchical linear modeling to examine the influence of the curriculum.

Centers for Learning and Teaching: Research to Identify Changes in Mathematics Education Doctoral Preparation and the Production of New Doctorates

This project will research the programmatic changes that resulted from the NSF investment in Centers for Learning and Teaching of Mathematics (CLT) at the 31 participating institutions. It will provide information on the core elements of doctoral preparation in mathematics education at the institutions and ways in which participation in the CLTs has changed their programs.

Lead Organization(s): 
Award Number: 
1434442
Funding Period: 
Fri, 08/01/2014 to Tue, 07/31/2018
Full Description: 

The quality of the mathematical education provided to teachers and ultimately to their students depends on the quality of teacher educators at the colleges and universities. For several decades, there has been a shortage of well-prepared mathematics teacher educators. Doctoral programs in mathematics education are the primary ways that these teacher educators learn the content and methods that they need to prepare teachers, but the quality of these programs varies and the number of qualified graduates has been insufficient to meet the demand.

This project will research the programmatic changes that resulted from the NSF investment in Centers for Learning and Teaching of Mathematics (CLT) at the 31 participating institutions. It will provide information on the core elements of doctoral preparation in mathematics education at the institutions and ways in which participation in the CLTs has changed their programs. It will also gather data on the number of doctorates in mathematics education from the CLT institutions prior to the establishment of the CLT and after their CLT ended. A comparison group of Doctoral granting institutions will be studied over the same time frame to determine the number of doctoral students graduated during similar time frames as the CLTs. Follow-up data from graduates of the CLTs will be gathered to identify programmatic strengths and weaknesses as graduates will be asked to reflect on how their doctoral preparation aligned with their current career path. The research questions are: What were the effects of CLTs on the production of new doctorates in mathematics education? What changes were made to doctoral programs in mathematics education by the CLT institutions? How well prepared were the CLT graduates for various career paths?

Bio-Sphere: Fostering Deep Learning of Complex Biology for Building our Next Generation's Scientists

The goal of this project is to help middle school students, particularly in rural and underserved areas, develop deep scientific knowledge and knowledge of the practices and routines of science. Research teams will develop an innovative learning environment called Bio-Sphere, which will foster learning of complex science issues through hands-on design and engineering.

Award Number: 
1418044
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

Today's citizens face profound questions in science. Preparing future generations of scientists is crucial if the United States is to remain competitive in a technology-focused economy. The biological sciences are of particular importance for addressing some of today's complex problems, such as sustainability and food production, biofuels, and carbon dioxide and its effect on our environment. Although knowledge in the life sciences is of critical importance, this is an area in which there are significantly fewer studies examining students' conceptions than in physics and chemistry. The goal of this project is to help middle school students, particularly in rural and underserved areas, develop deep scientific knowledge and knowledge of the practices and routines of science. A major strength of Bio-Sphere is the inclusion of hands-on design and engineering in biology, a field in which there are fewer instances of curricula that integrate engineering design at the middle school level. The units will enable an in-depth, cohesive understanding of science content, and Bio-Sphere will be disseminated nationally and internationally through proactive outreach to teachers as well as scholarly publications.

This project addresses the need to inculcate deep learning of complex science by bringing complex socio-scientific issues into middle school classrooms, and providing students with instructional materials that allow them to practice science as scientists do. Research teams will develop, iteratively refine and evaluate an innovative learning environment called Bio-Sphere. Bio-Sphere combines the strengths of hands-on design and engineering, engages students in the practices of science, and fosters learning of complex science issues, especially among underserved populations. Each Bio-Sphere unit presents a complex science issue in the form of a design challenge that students solve by conducting experiments, using visualizations in an electronic textbook, and connecting with the community. The units, aligned with the Next Generation Science Standards, provide greater coherence, continuity, and sustained instruction focused on uncovering and integrating key ideas over long periods of time. The project will follow a design-based research methodology. In Phase 1, the Bio-Sphere materials will be developed. Phase 2 will consist of studies in Wisconsin schools to generate existence proofs, i.e., examining enactments with respect to the designed objectives to understand how a design works. Phase 3 studies will focus on practical implementation: how to bring this innovative design to life in very different classroom contexts and without the everyday support of the design team, and will be conducted in rural schools in Alabama and North Carolina.

iSTEM: A Multi-State Longitudinal Study of the Effectiveness of Inclusive STEM High Schools

This is a quasi-experimental study of the effects of attending an inclusive STEM high school in three key geographic regions and comparing outcomes for students in these schools with those of their counterparts attending other types of schools in the same states. The study's focus is on the extent to which inclusive STEM high schools contribute to improved academic outcomes, interests in STEM careers, and expectations for post secondary study.

Lead Organization(s): 
Award Number: 
1817513
Funding Period: 
Sun, 09/01/2013 to Sat, 08/31/2019
Full Description: 

Researchers from SRI and George Washington University are studying the effectiveness of inclusive STEM high schools in three key geographic regions including Texas, North Carolina and Ohio. STEM schools continue to be an important policy area and test bed for one indication of what STEM education can accomplish under the most optimal conditions in which STEM is the focus of students' learning experiences. The President has called for the creation of an additional 1,000 STEM schools with relatively little evidence about the impact of such schools or the evidence of which configurations and elements of such schools are important. The study's focus is on the extent to which inclusive STEM high schools contribute to improved academic outcomes, interests in STEM careers, and expectations for post secondary study. The research study engages in implementation research to examine the elements of the STEM schools' design and implementation and other contextual factors, including state policies, which are associated with superior outcomes.

This is a quasi-experimental study of the effects of attending an inclusive STEM high school comparing outcomes for students in these schools with those of their counterparts attending other types of schools in the same states. The study includes all students in the 9th or 12th grade in the inclusive STEM high schools and students in samples of same-state comparison schools identified through propensity score matching. Data are collected longitudinally using student records and surveying students at regular intervals. The study follows the 12th grade students after graduation into postsecondary study and the workforce. The states identified in this study have the requisite administrative data systems to support the proposed study. By using a combination of data available in state-level data bases and new information obtained through project surveys, the researchers are identifying students who are matched not only on demographic variables and academic achievement before high school entry, but also on indicators of pre-existing interest and expectation such as self-efficacy and prior participation in informal STEM-related activities. Impacts on student achievement are analyzed separately for each state. Data on the elements of STEM schools are collected through teacher and administrator surveys and interviews. State STEM school history and policy data are collected through document analysis and interviews. The study utilizes hierarchical regression models, with separate models of each outcome measure and adjustments for tests of multiple comparisons. Student attrition is monitored and findings are examined to determine influence of attrition.

This project focuses on inclusive rather than selective STEM schools so that the population of students more typically represents the population of the students locally. The study provides a source of evidence about not only the effectiveness of STEM schools, but also contextual evidence of what works and for whom and under what conditions.

This project was previously funded under award # 1316920.

The Impact of Early Algebra on Students' Algebra-Readiness (Collaborative Research: Blanton)

In this project researchers are implementing and studying a research-based curriculum that was designed to help children in grades 3-5 prepare for learning algebra at the middle school level. Researchers are investigating the impact of a long-term, comprehensive early algebra experience on students as they proceed from third grade to sixth grade. Researchers are working to build a learning progression that describes how algebraic concepts develop and mature from early grades through high school.

Award Number: 
1219605
Funding Period: 
Mon, 10/01/2012 to Wed, 09/30/2015
Full Description: 

The Impact of Early Algebra on Students' Algebra-Readiness is a collaborative project at the University of Wisconsin and TERC, Inc. They are implementing and studying a research-based curriculum that was designed to help children in grades 3-5 prepare for learning algebra at the middle school level. Researchers are investigating the impact of a long-term, comprehensive early algebra experience on students as they proceed from third grade to sixth grade. Researchers are working to build a learning progression that describes how algebraic concepts develop and mature from early grades through high school. This study helps to build our knowledge about the piece of the progression that is just prior to entering middle school where many students begin formal instruction in algebra.

Building on previous research about early algebra learning, researchers will teach a curriculum that was carefully designed to reflect what we know about learning algebraic concepts. Previous research has shown that young children from very diverse backgrounds have the ability to construct algebraic ideas such as equality, representation, generalization, and functions. Researchers are collecting data about students' algebraic knowledge as well as arithmetical knowledge.

We know that the majority of students in the United States struggle with learning formal algebra. By studying the implementation of the research-based curriculum for an extended period of time, researcher's are learning about how algebraic ideas are connected and whether or not early instruction on algebraic ideas will help students learn more formal ideas in middle school.

Evaluation of the Sustainability and Effectiveness of Inquiry-Based Advanced Placement Science Courses: Evidence From an In-Depth Formative Evaluation and Randomized Controlled Study

This study examines the impact of the newly revised Advanced Placement (AP) Biology and Chemistry courses on students' understanding of and ability to utilize scientific inquiry, on students' confidence in engaging in college-level material, and on students’ enrollment and persistence in college STEM majors. The project provides estimates of the impact of students' AP-course taking on their progress into postsecondary educational experiences and their intent to continue to prepare to be future engineers and scientists.

Award Number: 
1220092
Funding Period: 
Sat, 09/15/2012 to Wed, 08/31/2016
Full Description: 

This study examines the impact of the newly revised Advanced Placement (AP) Biology and Chemistry courses on students' understanding of and ability to apply scientific inquiry, on students' confidence in successfully engaging in college-level material, and on students enrollment and persistence in college STEM majors. AP Biology and Chemistry courses represent an important educational program that operates at a large scale across the country. The extent to which the AP curricula vary in implementation across the schools in the study is also examined to determine the range of students' opportunity to learn the disciplinary content and the knowledge and skills necessary to engage in inquiry in science. Schools that are newly implementing AP courses are participants in this research and the challenges and successes that they experience are also a component of the research plan. Researchers at the University of Washington, George Washington University and SRI International are conducting the study.

The research design for this study includes both formative components and a randomized control experiment. Formative elements include observations, interviews and surveys of teachers and students in the AP courses studied. The experimental design includes the random assignment of students to the AP offered and follows the performances of the treatment and control students in two cohorts into their matriculation into postsecondary educational experiences. Surveys measure students' experiences in the AP courses, their motivations to study AP science, the level of stress they experience in their high school coursework and their scientific inquiry skills and depth of disciplinary knowledge. The study examines the majors chosen by those students who enter into colleges and universities to ascertain the extent to which they continue in science and engineering.

This project informs educators about the challenges and successes schools encounter when they expand access to AP courses. The experiences of the teachers who will be teaching students with variable preparation inform future needs for professional development and support. The project provides estimates of the impact of students' AP-course taking on their progress into postsecondary educational experiences and their intent to continue to prepare to be future engineers and scientists. It informs policy efforts to improve the access to more rigorous advanced courses in STEM and provides strong experimental evidence of the impact of AP course taking. The project has the potential to demonstrate to educational researchers how to study an educational program that operates at scale.

Researching the Efficacy of the Science and Literacy Academy Model (Collaborative Research: Osborne)

This project is studying three models of professional development (PD) to test the efficacy of a practicum for grade 3-5 in-service teachers organized in three cohorts of 25. There will be 75 teachers and their students directly impacted by the project. Additional impacts of the project are research results and professional development materials, including a PD implementation guide and instructional videos.

Lead Organization(s): 
Award Number: 
1220666
Funding Period: 
Wed, 08/15/2012 to Sun, 07/31/2016
Full Description: 

This award is doing a research study of three models of professional development (PD) to test the efficacy of a practicum for grade 3-5 in-service teachers organized in three cohorts of 25. Model 1 is a one-week institute based on classroom discourse practices and a 2-week practicum (cohort 1). Model 2 is the one-week institute (cohort 2). Model 3 is a "business as usual" model (cohort 3) based on normal professional development provided by the school district. Cohorts 1 and 2 experience the interventions in year 1 with four follow-up sessions in each of years 2 and 3. In year 4 they receive no PD, but are being observed to see if they sustain the practices learned. Cohort 3 receives no treatment in years 1 and 2, but participates in a revised version of the institute plus practicum in year 3 with four follow up sessions in year 4. The Lawrence Hall of Science provides the professional development, and Stanford University personnel are conducting the research. The teachers come from the Oakland Unified School District. Science content is the GEMS Ocean Sciences Sequence.

There are 3 research questions;

1. In what ways do practicum-based professional development models influence science instructional practice?

2. What differences in student outcomes are associated with teachers' participation in the different PD programs?

3. Is the impact of the revised PD model different from the impact of the original model?

This is a designed-based research model. Teacher data is based on interviews on beliefs about teaching and the analysis of video tapes of their practicum and classroom performance using the Discourse in Inquiry Science Classrooms instrument. Student data is based on the GEMS unit pre- and post-tests and the California Science Test for 5th graders. Multiple analyses are being conducted using different combinations of the data from 8 scales across 4 years.

There will be 75 teachers and their students directly impacted by the project. Additional impacts of the project are research results and professional development materials, including a PD implementation guide and instructional videos. These will be presented in publications and conference presentations and be posted on linked websites at the Lawrence Hall of Science and the Center to Support Excellence in Teaching at Stanford University.

Pages

Subscribe to Effectiveness