Case Study

Constructing and Critiquing Arguments in Middle School Science Classrooms: Supporting Teachers with Multimedia Educative Curriculum Materials

This project is developing Earth and Space Science multimedia educative curriculum materials (MECMs) and a system to facilitate teachers' learning and beliefs of scientific argumentation. The project is investigating the impact of the MECMs on teachers' beliefs about scientific argumentation and their related pedagogical content knowledge. The overarching research question focuses on how can multimedia educative curriculum materials provide support to middle school science teachers in implementing standards for constructing and critiquing arguments.

Project Email: 
Award Number: 
1119584
Funding Period: 
Thu, 09/01/2011 to Sun, 08/31/2014
Project Evaluator: 
Naomi Hupert
Full Description: 

This project between Lawrence Hall of Science and Boston College is developing Earth and Space Science multimedia educative curriculum materials (MECM) and a system to facilitate teachers' learning and beliefs of scientific argumentation. The MECMs include videos, voice-over narratives, diagrammatic representations, images of student writings, and text. The PIs are investigating the impact of the MECMS on teachers' beliefs about scientific argumentation and their related pedagogical content knowledge. The overarching research question, with four sub questions, focuses on how can multimedia educative curriculum materials provide support to middle school science teachers in implementing standards for constructing and critiquing arguments. The four sub questions are: What factors impact teachers' implementation of argumentation instruction in the classroom? How can MECMs be designed to positively impact teachers' beliefs and their pedagogical content knowledge (PCK) about argumentation? What is the relationship between teachers' beliefs about the value of argumentation and their implementation of argumentation in the classroom? What impact do MECMs have on teachers' beliefs and PCK?

A mixed method approach is being used to assess teachers' beliefs and pedagogical content knowledge. The PIs are developing and pilot testing teachers' beliefs about scientific argumentation. They will use an iterative design process for the MECMs that will involve 50 teachers. Twenty-five phone interviews will be conducted to investigate factors that impact teachers' implementations of scientific argumentation. Three iterative cycles of design and testing include focus groups, a pilot of the MECMs in six classrooms, and a national field test of 30 classrooms. One hundred teachers will field test the assessment followed by collection of six case studies and data analyses. The project's formative and summative evaluations include monitoring and providing feedback for all activities, and assessments of program implementation and impact.

Teachers need support using field tested multimedia educative materials (MECMs) in learning and delivering science content using a scientific argumentation process. By delivering and engaging the teaching and learning process through iterative design of Earth and Space Science multimedia educative curriculum materials, this project would provide, if successful, teachers and students with the necessary literacy and knowledge about scientific argumentation. The MECMs and approach has the potential for broad implementation in middle schools and beyond for delivering Earth and Space science material to support and teach scientific argumentation.

InterLACE: Interactive Learning and Collaboration Environment

This project designs, constructs, and field-tests a web-based, online collaborative environment for supporting the teaching and learning of inquiry-based high school physics. Based on an interactive digital workbook environment, the team is customizing the platform to include scaffolds and other supports for learning physics, fostering interaction and collaboration within the classroom, and facilitating a design-based approach to scientific experiments.

Lead Organization(s): 
Award Number: 
1119321
Funding Period: 
Thu, 09/01/2011 to Sat, 08/31/2013
Full Description: 

This project, under the Tufts University Center for Engineering Education and Outreach (CEEO) designs, constructs, and field-tests a web-based, online collaborative environment for supporting the teaching and learning of inquiry-based high school physics. Based on prior NSF-funded work on RoboBooks, an interactive digital workbook environment, the team is customizing the platform to include scaffolds and other supports for learning physics, fostering interaction and collaboration within the classroom, and facilitating a design-based approach to scientific experiments. The InterLACE team hypothesizes that technology seamlessly integrating physics content and process skills within a classroom learning activity will provide a wide variety of student benefits, ranging from improved learning outcomes and increased content knowledge to gains in attitudinal and social displays as well.

The hypothesis for this work is based on research that indicates teachers believe proper implementation of design-based, inquiry projects are time consuming and can be difficult to manage and facilitate in classrooms without great scaffolding or other supports. Using design-based research with a small number of teachers and students, the PIs iteratively develop the system and supporting materials and generate a web-based implementation that supports students through the various stages of design inquiry. A quasi-experimental trial in the final years of the project is used to determine the usability of the technology and efficacy of the system in enhancing teaching and learning. Through the tools and activities developed, the researchers anticipate showing increases in effective inquiry learning and enhanced accessibility to meet the needs of diverse learners and teachers, leading to changes in classroom practice.

Through this project the PIs (1) gain insights that will enable them to refine the InterLACE platform so it can be implemented and brought to scale in the near terms as a support for design-based inquiry science projects, and (2) advance theory, design and practice to support the design of technology-based learning environments, and (3) understand how connecting students? hypotheses, ideas, and data impacts their learning of physics content and scientific inquiry skills.

Project ATOMS: Accomplished Elementary Teachers of Mathematics and Science

The project is studying the impact of the mathematics and science intensive pre-service preparation program for elementary school teachers.  The project includes assessments of pre-service teachers' math and science content, teacher performance, self-report surveys, and teacher interviews. Each of the study dimensions (Knowledge Dimension, Teaching Performance, and Perspectives on the Program) will be assessed at three time points across this longitudinal study, providing a model for elementary teacher development of STEM teaching.

Partner Organization(s): 
Award Number: 
1118894
Funding Period: 
Thu, 09/01/2011 to Sat, 08/31/2019
Full Description: 

The project is studying the impact of the mathematics and science intensive pre-service preparation program for elementary school teachers at North Carolina State University called the Accomplished Elementary Teachers of Mathematics and Science (ATOMS). Faculty in NCSU's Department of Elementary Education, researchers at the Duke University Sanford School of Public Policy's Education Research Data Center and the NC State College Professional Education Office are involved in conducting this project.

The project includes assessments of pre-service teachers' math and science content, teacher performance, self-report surveys, and teacher interviews. Researchers are also tracking participants' perspectives on the program and comparing knowledge dimensions and teaching performance of a sub-sample of ATOMS teachers to a similar group of non-ATOMS teachers. Each of the study dimensions (Knowledge Dimension, Teaching Performance, and Perspectives on the Program) will be assessed at three time points across this longitudinal study, providing a model for elementary teacher development of STEM teaching.

The study has potential to advance current understanding regarding teacher preparation, especially in terms of supporting elementary teachers' instruction in science and math. The project is also innovative and potentially transformative by asking interesting and pertinent questions of how teachers can affect the learning of their students. Besides generating new knowledge, this project also has the potential to impact STEM education research. The ATOMS pre-service teacher preparation program may serve as a model for effective pre-service teacher education across the nation if the researchers can clearly demonstrate the effect of participating in the program in changing teachers' knowledge, attitudes, and skills, as well as their students' achievement. Investigators propose the dissemination of findings to both K-12 audiences and institutions of higher education. Additionally, key findings will be bulleted for policy makers in brief reports or brochures sent to deans of Colleges of Education nationwide, highlighting recommendations based on the findings.

An Innovative Approach to Earth Science Teacher Preparation: Uniting Science, Informal Science Education, and Schools to Raise Student Achievement

The American Museum of Natural History in New York City, in partnership with New York University, and in collaboration with five high-needs schools, is developing, implementing, and researching a five-year pilot Master of Arts in Teaching (MAT) program in Earth Science. The program is delivered by the Museum's scientific and education teams and its evaluation covers aspects of the program from recruitment to first year of teaching.

Project Email: 
Award Number: 
1119444
Funding Period: 
Thu, 09/01/2011 to Thu, 08/31/2017
Project Evaluator: 
David Silvernail, Center for Education and Policy, University of Southern Maine
Full Description: 

The American Museum of Natural History (AMNH), in collaboration with New York University's Institute for Education and Social Policy and the University of Southern Maine Center for Evaluation and Policy, will develop and evaluate a new teacher education program model to prepare science teachers through a partnership between a world class science museum and high need schools in metropolitan New York City (NYC). This innovative pilot residency model was approved by the New York State (NYS) Board of Regents as part of the state’s Race To The Top award. The program will prepare a total of 50 candidates in two cohorts (2012 and 2013) to earn a Board of Regents-awarded Masters of Arts in Teaching (MAT) degree with a specialization in Earth Science for grades 7-12. The program focuses on Earth Science both because it is one of the greatest areas of science teacher shortages in urban areas and because AMNH has the ability to leverage the required scientific and educational resources in Earth Science and allied disciplines, including paleontology and astrophysics.

The proposed 15-month, 36-credit residency program is followed by two additional years of mentoring for new teachers. In addition to a full academic year of residency in high-needs public schools, teacher candidates will undertake two AMNH-based clinical summer residencies; a Museum Teaching Residency prior to entering their host schools, and a Museum Science Residency prior to entering the teaching profession. All courses will be taught by teams of doctoral-level educators and scientists.

The project’s research and evaluation components will examine the factors and outcomes of a program offered through a science museum working with the formal teacher preparation system in high need schools. Formative and summative evaluations will document all aspects of the program. In light of the NYS requirement that the pilot program be implemented in high-need, low-performing schools, this project has the potential to engage, motivate and improve the Earth Science achievement and interest in STEM careers of thousands of students from traditionally underrepresented populations including English language learners, special education students, and racial minority groups. In addition, this project will gather meaningful data on the role science museums can play in preparing well-qualified Earth Science teachers. The research component will examine the impact of this new teacher preparation model on student achievement in metropolitan NYC schools. More specifically, this project asks, "How do Earth Science students taught by first year AMNH MAT Earth Science teachers perform academically in comparison with students taught by first year Earth Science teachers not prepared in the AMNH program?.”

Multiple Instrumental Case Studies of Inclusive STEM-Focused High Schools: Opportunity Structures for Preparation and Inspiration (OSPrl)

The aim of this project is to examine opportunity structures provided to students by inclusive STEM-focused high schools, with an emphasis on studying schools that serve students from underrepresented groups. The project is studying inclusive STEM-focused high schools across the United States to determine what defines them. The research team initially identified ten candidate critical components that define STEM-focused high schools and is refining and further clarifying the critical components through the research study.

Lead Organization(s): 
Award Number: 
1118851
Funding Period: 
Thu, 09/01/2011 to Mon, 08/31/2015
Full Description: 

The aim of this project is to examine opportunity structures provided to students by inclusive STEM-focused high schools, with an emphasis on studying schools that serve students from underrepresented groups. In contrast to highly selective STEM-focused schools that target students who are already identified as gifted and talented in STEM, inclusive STEM-focused high schools aim to develop new sources of STEM talent, particularly among underrepresented minority students, to improve workforce development and prepare STEM professionals. A new NRC report, Successful K-12 STEM Education (2011), identifies areas in which research on STEM-focused schools is most needed. The NRC report points out the importance of providing opportunities for groups that are underrepresented in the sciences, especially Blacks, Hispanics, and low-income students who disproportionately fall out of the high-achieving group in K-12 education. This project responds specifically to the call for research in the NRC report and provides systematic data to define and clarify the nature of such schools. 

The project is studying inclusive STEM-focused high schools across the United States to determine what defines them. The research team initially identified ten candidate critical components that define STEM-focused high schools and is refining and further clarifying the critical components through the research study. The first phase of the study is focusing on 12 well-established and carefully planned schools with good reputations and strong community and business support, in order to capture the critical components as intended and implemented. Case studies of these high-functioning schools and a cross-case analysis using a set of instruments for gauging STEM design and implementation are contributing toward building a theory of action for such schools that can be applied more generally to STEM education. The second phase of the study involves selecting four school models for further study, focusing on student-level experiences and comparing student outcomes against comprehensive schools in the same district. Research questions being studied include: 1) Is there a core set of likely critical components shared by well-established, promising inclusive STEM-focused high schools? Do other components emerge from the study? 2) How are the critical components implemented in each school? 3) What are the contextual affordances and constraints that influence schools' designs, their implementation, and student outcomes? 4) How do student STEM outcomes in these schools compare with school district and state averages? 5) How do four promising such schools compare with matched comprehensive high schools within their respective school districts, and how are the critical components displayed? 6) From the points of view of students underrepresented in STEM fields, how do education experiences at the schools and their matched counterparts compare? And 7) How do student outcomes compare?

The research uses a multiple instrumental case study design in order to describe and compare similar phenomena. Schools as critical cases are being selected through a nomination process by experts, followed by screening and categorization according to key design dimensions. Data sources include school documents and public database information; a survey, followed by telephone interviews that probe for elaborated information, to provide a systematic overview of the candidate components; on-site visitations to each school provide data on classroom observations at the schools; interviews with students, teachers and administrators in focus groups; and discussions with critical members of the school community that provide unique opportunities to learn such as mentors, business leaders, and members of higher education community that provide outside of school learning experiences. The project is also gathering data on a variety of school-level student outcome indicators, and is tracking the likely STEM course trajectories for students, graduation rates, and college admission rates for students in the inclusive STEM-focused schools, as compared to other schools in the same jurisdiction. Analysis of the first phase of the study aims to develop rich descriptions that showcase characteristics of the schools, using axial and open coding, to determine a theory of action that illustrates interconnections among context, design, implementation, and outcome elements. Analysis of the second phase of the study involves similar processes on four levels: school, student, databases, and a synthesis of the three. Evaluation of the project consists of an internal advisory board and an external advisory board, both of which provide primarily formative feedback on research procedures.

Research findings, as well as case studies, records of instrument and rubric development and use, annual reports, and conference proposals and papers are being provided on a website, in order to provide an immediate and ongoing resource for education leaders, researchers and policymakers to learn about research on these schools and particular models. An effort is also being made to give voice to the experiences of high school students from the four pairs of high schools studied in the second phase of the study. Findings are also being disseminated by more traditional means, such as papers in peer-reviewed journals and conference presentations.

Developing Teaching Expertise in K-5 Mathematics

This project designs materials and an accompanying support system to enable the development of expertise in the teaching of mathematics at the elementary level. The project has four main components: online professional development modules; practice-based assessments; resources for facilitators; and web-based technologies to deliver module content to diverse settings. Three modules are being developed and focus on fractions, reasoning and explanation, and geometry. Each module is organized into ten 1.5 hour sessions.

Project Email: 
Award Number: 
1118745
Funding Period: 
Thu, 09/01/2011 to Fri, 08/31/2018
Project Evaluator: 
American Institutes for Research
Full Description: 

Developers and researchers at the University of Michigan and the University of Denver are engaged in a project to design materials and an accompanying support system to enable the development of expertise in the teaching of mathematics at the elementary level. The project has four main components: a set of online professional development modules; practice-based assessments; a set of resources for facilitators; and web-based technologies to deliver module content to diverse settings. Three modules are planned: one focused on fractions and one focused on reasoning and explanation designed by Deborah Ball, Hyman Bass and the University of Michigan development team; and one on geometry developed by Douglas Clements and Julie Sarama at the University of Denver. Each module is organized into ten 1.5 hour sessions. 

Each module goes through a two-year design and development process that includes initial design, piloting, revision, and dissemination. Modules are piloted in a variety of settings, including university based courses for practicing teachers and district based in-service activities. These contexts include face-to-face professional development, real-time distance learning, and combinations of the two. Data are collected on participant engagement with the modules, on teacher classroom practice, and on mathematical knowledge for teaching.

The modules and associated materials will be widely available and will be free to schools. The materials can be imported into any learning management system, such as Blackboard, Moodle, and others.

Continuous Learning and Automated Scoring in Science (CLASS)

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items into the "Web-based Inquiry Science Environment" (WISE) program.

Award Number: 
1119670
Funding Period: 
Thu, 09/01/2011 to Mon, 08/31/2015
Full Description: 

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items (i.e., short essays, science narratives, concept mapping, graphing problems, and virtual experiments) into the "Web-based Inquiry Science Environment" (WISE) program. WISE is an online science-inquiry curricula that supports deep understanding through visualization of processes not directly observable, virtual experiments, graphing results, collaboration, and response to prompts for explanations. In partnership with Educational Testing Services (ETS), project goals are: (1) to develop five automated inquiry assessment activities that capture students' abilities to integrate their ideas and form coherent scientific arguments; (2) to customize WISE by incorporating automated scores; (3) to investigate how students' systematic feedback based on these scores improve their learning outcomes; and (4) to design professional development resources to help teachers use scores to improve classroom instruction, and administrators to make better informed decisions about teacher professional development and inquiry instruction. The project targets general science (life, physical, and earth) in three northern California school districts, five middle schools serving over 4,000 6th-8th grade students with diverse cultural and linguistic backgrounds, and 29 science teachers. It contributes to increase opportunities for students to improve their science achievement, and for teachers and administrators to make efficient, evidence-based decisions about high-quality teaching and learning.

A key research question guides this effort: How automated scoring of inquiry assessments can increase success for diverse students, improve teachers' instructional practices, and inform administrators' decisions about professional development, inquiry instruction, and assessment? To develop science inquiry assessment activities, scoring written responses include semantic, syntax, and structure of meaning analyses, as well as calibration of human-scored items with a computer-scoring system through the c-rater--an ETS-developed cyber learning technology. Validity studies are conducted to compare automated scores with human-scored items, teacher, district, and state scores, including sensitivity to the diverse student population. To customize the WISE curriculum, the project modifies 12 existing units and develops nine new modules. To design adaptive feedback to students, comparative studies explore options for adaptive guidance and test alternatives based on automated scores employing linear models to compare student performance across randomly assigned guidance conditions; controlling for covariates, such as prior science scores, gender, and language; and grouping comparison studies. To design teacher professional development, synthesis reports on auto-scored data are created to enable them to use evidence to guide curricular decisions, and comments' analysis to improve feedback quality. Workshops, classroom observations, and interviews are conducted to measure longitudinal teachers' change over time. To empower administrators' decision making, special data reports, using-evidence activities, individual interviews, and observation of administrators' meetings are conducted. An advisory board charged with project evaluation addresses both formative and summative aspects.

A research-informed model to improve science teaching and learning at the middle school level through cyber-enabled assessment is the main outcome of this effort. A total of 21 new, one- to three-week duration standards-based science units, each with four or more automatically scored items, serve as prototypes to improve students' performance, teachers' instructional approaches, and administrators' school policies and practices.

School Organization and Science Achievement: Organization and Leadership Influences on Equitable Student Performance (Collaborative Research: Settlage)

This project will document factors explaining variations in science achievement across schools enrolling ethnically and linguistically diverse students. The research question is: what leadership and organizational features at the school level are associated with mitigating science achievement gaps? At the conclusion of the five-year project, the findings will take the form of recommendations about leadership practices and school organization that can be implemented in other school settings.

Award Number: 
1119349
Funding Period: 
Fri, 07/01/2011 to Sun, 06/30/2013
Project Evaluator: 
Katherine Paget, Education Development Center, Inc. (EDC)
Full Description: 

The School Organization and Science Achievement (SOSA) Project will document factors explaining variations in science achievement across schools enrolling ethnically and linguistically diverse students. The research question is: what leadership and organizational features at the school level are associated with mitigating science achievement gaps? Previous school effectiveness studies demonstrate school leadership and social capital influencing student achievement; the SOSA project is unique with its focus on science achievement. Researchers at the University of Connecticut and the University of South Florida St. Petersburg, in collaboration with school districts in their respective states, will identify school leadership practices that can be connected with reductions in achievement gaps related to student ethnicity, English fluency, and social status. At the conclusion of the five-year project, the findings will take the form of recommendations about leadership practices and school organization that can be implemented in other school settings.

The project uses a mixed methods design by combining statistical modeling and qualitative data. Multiple regression analyses highlight those schools populated by fifth graders that have greater or lesser achievement gaps in science. Using social capital theory (i.e., school norms, communication channels, and trustworthiness) comparisons of positive and negative outlier schools will be made via interviews of building principals, classroom teachers and community representatives. The expectation is that schools providing more equitable science experiences to all students will exhibit stronger social capital compared to buildings with disparities in science test scores across demographic categories. These insights will be supplemented by multilevel structural equation modeling to determine the strength of association between various school climate measures (e.g., teacher-to-principal trust, correspondence between teacher and principal perceptions of leadership, and school/community ties) and science achievement as measured by statewide fifth grade science tests. In addition, growth analyses will be used to detect shifts over time and provide insights about the links between policy changes or leadership adjustments, inasmuch as science achievement gaps are affected.

By working with 150 schools in two states, this collaborative research project is designed to generate findings applicable in other school systems. Particularly in settings where science achievement gaps are large, and especially when such gaps vary between schools even when the student populations are similar, the findings from this study will have practical leadership implications. Expertise in this project includes science education, educational leadership, and statistical modeling. This complementary combination increases the depth of the project's efforts along with expanding its potential impacts. Key questions addressed by this project include: to what extent is leadership in science similar to or different from leadership in other subject areas? how do variations in leadership design (e.g., top-down versus distributed leadership) contribute to reductions in science achievement gaps? to what degree can effective leadership mitigate other factors that exacerbate the challenges of providing high quality science learning experiences for every child? Findings will be disseminated via the SOSA Project website, along with leadership development strategies. Deliverables include templates to replicate the study, case studies for professional development, and strategies for supporting the development of science teacher-leaders.

CAREER: Engaging Elementary Students in Data Analysis Through Study of Physical Activities

This project is investigating the learning that can take place when elementary school students are directly involved in the collection, sense-making, and analysis of real, personally-meaningful data sets. The hypotheses of this work are that by organizing elementary statistics instruction around the study of physical activities, students will have greater personal engagement in data analysis processes and that students will also develop more robust understandings of statistical ideas.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1054280
Funding Period: 
Fri, 07/01/2011 to Sat, 06/30/2018
Full Description: 

This CAREER awardee at Utah State University is investigating the learning that can take place when elementary school students are directly involved in the collection, sense-making, and analysis of real, personally-meaningful data sets. The project responds to increasing attention to data collection and analysis in elementary grades and aims to make important contributions to the knowledge base on effective approaches to these topics. The hypotheses of this work are that by organizing elementary statistics instruction around the study of physical activities, students will have greater personal engagement in data analysis processes and that students will also develop more robust understandings of statistical ideas. Students and teachers from fifth grade classrooms from several elementary schools from northern Utah, are participating in the project. This work is co-funded by the EPSCoR program.

Statistics topics include measures of center and variation. Students use pedometers, heart rate monitors, other probeware, and the TinkerPlots software. The research team investigates the influence of personal ownership and relationships to data on students' understanding of learning of elementary statistics concepts and their ability to analyze data. The research involves multi-year clinical interviews and video-recorded classroom design experiments.

Research results are expected to be published in appropriate journals and are expected to be presented at professional meetings. Lesson plans and student instructional materials related to physical activity, measures of center, and data distributions are made available for use in partner elementary schools.

CAREER: Noticing and Capitalizing on Important Mathematical Moments in Instruction

This project investigates the outcomes of a teacher education model designed to foster prospective mathematics teachers' abilities to notice and capitalize on important mathematical moments in instruction. The project engages prospective teachers in research-like analysis of unedited teacher-perspective classroom video early in their teacher education coursework in order to help them learn to identify, assess the mathematical potential of, and respond to important student ideas and insights that arise during instruction.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1052958
Funding Period: 
Fri, 04/15/2011 to Sat, 03/31/2012
Full Description: 

This CAREER awardee at Michigan Technological University is investigating the outcomes of a teacher education model designed to foster prospective mathematics teachers' abilities to notice and capitalize on important mathematical moments in instruction. The researcher engages prospective teachers in research-like analysis of unedited teacher-perspective classroom video early in their teacher education coursework in order to help them learn to identify, assess the mathematical potential of, and respond to important student ideas and insights that arise during instruction.

The research is based on a quasi-experimental design and involves three cohorts of prospective teachers. Practicing teachers from local schools collaborate with the research team. The data collected consists of classroom video. The video is coded and analyzed using Studiocode, which allows for real-time coding and for multiple users to code and annotate video segments.

The research findings are integrated into the institution's teacher education program and are also disseminated more broadly through publication and presentations at professional meetings.

Pages

Subscribe to Case Study