Effectiveness

An Innovative Approach to Earth Science Teacher Preparation: Uniting Science, Informal Science Education, and Schools to Raise Student Achievement

The American Museum of Natural History in New York City, in partnership with New York University, and in collaboration with five high-needs schools, is developing, implementing, and researching a five-year pilot Master of Arts in Teaching (MAT) program in Earth Science. The program is delivered by the Museum's scientific and education teams and its evaluation covers aspects of the program from recruitment to first year of teaching.

Project Email: 
Award Number: 
1119444
Funding Period: 
Thu, 09/01/2011 to Thu, 08/31/2017
Project Evaluator: 
David Silvernail, Center for Education and Policy, University of Southern Maine
Full Description: 

The American Museum of Natural History (AMNH), in collaboration with New York University's Institute for Education and Social Policy and the University of Southern Maine Center for Evaluation and Policy, will develop and evaluate a new teacher education program model to prepare science teachers through a partnership between a world class science museum and high need schools in metropolitan New York City (NYC). This innovative pilot residency model was approved by the New York State (NYS) Board of Regents as part of the state’s Race To The Top award. The program will prepare a total of 50 candidates in two cohorts (2012 and 2013) to earn a Board of Regents-awarded Masters of Arts in Teaching (MAT) degree with a specialization in Earth Science for grades 7-12. The program focuses on Earth Science both because it is one of the greatest areas of science teacher shortages in urban areas and because AMNH has the ability to leverage the required scientific and educational resources in Earth Science and allied disciplines, including paleontology and astrophysics.

The proposed 15-month, 36-credit residency program is followed by two additional years of mentoring for new teachers. In addition to a full academic year of residency in high-needs public schools, teacher candidates will undertake two AMNH-based clinical summer residencies; a Museum Teaching Residency prior to entering their host schools, and a Museum Science Residency prior to entering the teaching profession. All courses will be taught by teams of doctoral-level educators and scientists.

The project’s research and evaluation components will examine the factors and outcomes of a program offered through a science museum working with the formal teacher preparation system in high need schools. Formative and summative evaluations will document all aspects of the program. In light of the NYS requirement that the pilot program be implemented in high-need, low-performing schools, this project has the potential to engage, motivate and improve the Earth Science achievement and interest in STEM careers of thousands of students from traditionally underrepresented populations including English language learners, special education students, and racial minority groups. In addition, this project will gather meaningful data on the role science museums can play in preparing well-qualified Earth Science teachers. The research component will examine the impact of this new teacher preparation model on student achievement in metropolitan NYC schools. More specifically, this project asks, "How do Earth Science students taught by first year AMNH MAT Earth Science teachers perform academically in comparison with students taught by first year Earth Science teachers not prepared in the AMNH program?.”

School Structure and Science Success: Organization and Leadership Influences on Student Achievement (Collaborative Research: Butler)

This project will document factors explaining variations in science achievement across schools enrolling ethnically and linguistically diverse students. The research question is: what leadership and organizational features at the school level are associated with mitigating science achievement gaps? Researchers, in collaboration with school districts, will identify school leadership practices that can be connected with reductions in achievement gaps related to student ethnicity, English fluency, and social status.

Partner Organization(s): 
Award Number: 
1338512
Funding Period: 
Fri, 02/15/2013 to Sat, 06/30/2018
Full Description: 

The School Organization and Science Achievement (SOSA) Project will document factors explaining variations in science achievement across schools enrolling ethnically and linguistically diverse students. The research question is: what leadership and organizational features at the school level are associated with mitigating science achievement gaps? Previous school effectiveness studies demonstrate school leadership and social capital influencing student achievement; the SOSA project is unique with its focus on science achievement. Researchers at the University of Connecticut and the University of South Florida St. Petersburg, in collaboration with school districts in their respective states, will identify school leadership practices that can be connected with reductions in achievement gaps related to student ethnicity, English fluency, and social status. At the conclusion of the five-year project, the findings will take the form of recommendations about leadership practices and school organization that can be implemented in other school settings.

The project uses a mixed methods design by combining statistical modeling and qualitative data. Multiple regression analyses highlight those schools populated by fifth graders that have greater or lesser achievement gaps in science. Using social capital theory (i.e., school norms, communication channels, and trustworthiness) comparisons of positive and negative outlier schools will be made via interviews of building principals, classroom teachers and community representatives. The expectation is that schools providing more equitable science experiences to all students will exhibit stronger social capital compared to buildings with disparities in science test scores across demographic categories. These insights will be supplemented by multilevel structural equation modeling to determine the strength of association between various school climate measures (e.g., teacher-to-principal trust, correspondence between teacher and principal perceptions of leadership, and school/community ties) and science achievement as measured by statewide fifth grade science tests. In addition, growth analyses will be used to detect shifts over time and provide insights about the links between policy changes or leadership adjustments, inasmuch as science achievement gaps are affected.

By working with 150 schools in two states, this collaborative research project is designed to generate findings applicable in other school systems. Particularly in settings where science achievement gaps are large, and especially when such gaps vary between schools even when the student populations are similar, the findings from this study will have practical leadership implications. Expertise in this project includes science education, educational leadership, and statistical modeling. This complementary combination increases the depth of the project's efforts along with expanding its potential impacts. Key questions addressed by this project include: to what extent is leadership in science similar to or different from leadership in other subject areas? how do variations in leadership design (e.g., top-down versus distributed leadership) contribute to reductions in science achievement gaps? to what degree can effective leadership mitigate other factors that exacerbate the challenges of providing high quality science learning experiences for every child? Findings will be disseminated via the SOSA Project website, along with leadership development strategies. Deliverables include templates to replicate the study, case studies for professional development, and strategies for supporting the development of science teacher-leaders.

This project was previously funded under award # 1119359.

Continuous Learning and Automated Scoring in Science (CLASS)

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items into the "Web-based Inquiry Science Environment" (WISE) program.

Award Number: 
1119670
Funding Period: 
Thu, 09/01/2011 to Mon, 08/31/2015
Full Description: 

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items (i.e., short essays, science narratives, concept mapping, graphing problems, and virtual experiments) into the "Web-based Inquiry Science Environment" (WISE) program. WISE is an online science-inquiry curricula that supports deep understanding through visualization of processes not directly observable, virtual experiments, graphing results, collaboration, and response to prompts for explanations. In partnership with Educational Testing Services (ETS), project goals are: (1) to develop five automated inquiry assessment activities that capture students' abilities to integrate their ideas and form coherent scientific arguments; (2) to customize WISE by incorporating automated scores; (3) to investigate how students' systematic feedback based on these scores improve their learning outcomes; and (4) to design professional development resources to help teachers use scores to improve classroom instruction, and administrators to make better informed decisions about teacher professional development and inquiry instruction. The project targets general science (life, physical, and earth) in three northern California school districts, five middle schools serving over 4,000 6th-8th grade students with diverse cultural and linguistic backgrounds, and 29 science teachers. It contributes to increase opportunities for students to improve their science achievement, and for teachers and administrators to make efficient, evidence-based decisions about high-quality teaching and learning.

A key research question guides this effort: How automated scoring of inquiry assessments can increase success for diverse students, improve teachers' instructional practices, and inform administrators' decisions about professional development, inquiry instruction, and assessment? To develop science inquiry assessment activities, scoring written responses include semantic, syntax, and structure of meaning analyses, as well as calibration of human-scored items with a computer-scoring system through the c-rater--an ETS-developed cyber learning technology. Validity studies are conducted to compare automated scores with human-scored items, teacher, district, and state scores, including sensitivity to the diverse student population. To customize the WISE curriculum, the project modifies 12 existing units and develops nine new modules. To design adaptive feedback to students, comparative studies explore options for adaptive guidance and test alternatives based on automated scores employing linear models to compare student performance across randomly assigned guidance conditions; controlling for covariates, such as prior science scores, gender, and language; and grouping comparison studies. To design teacher professional development, synthesis reports on auto-scored data are created to enable them to use evidence to guide curricular decisions, and comments' analysis to improve feedback quality. Workshops, classroom observations, and interviews are conducted to measure longitudinal teachers' change over time. To empower administrators' decision making, special data reports, using-evidence activities, individual interviews, and observation of administrators' meetings are conducted. An advisory board charged with project evaluation addresses both formative and summative aspects.

A research-informed model to improve science teaching and learning at the middle school level through cyber-enabled assessment is the main outcome of this effort. A total of 21 new, one- to three-week duration standards-based science units, each with four or more automatically scored items, serve as prototypes to improve students' performance, teachers' instructional approaches, and administrators' school policies and practices.

Implementing the Mathematical Practice Standards: Enhancing Teachers' Ability to Support the Common Core State Standards

This is a four-year project that is producing materials designed to help teachers see how the mathematical practices described in the Common Core State Standards for mathematics can be implemented in mathematics instruction. The goal of the improved instruction is to help students adopt and value these critical mathematical practices.

Award Number: 
1119163
Funding Period: 
Mon, 08/01/2011 to Tue, 07/31/2012
Full Description: 

The Implementing Mathematical Practices Standards (IMPS) is a four-year project that is producing materials designed to help teachers see how the mathematical practices described in the Common Core State Standards for mathematics can be implemented in mathematics instruction. The goal of the improved instruction is to help students adopt and value these critical mathematical practices. Researchers at the Education Development Center are developing videos and print materials that exemplify the mathematical practices and are working with teachers in grades 5-10 to help them use the materials effectively. The research questions of the project are focused on what features of the materials are most helpful to teachers and what professional development characteristics facilitate implementation of the mathematics practices in classroom instruction. The external evaluation of the project is being conducted by evaluators at TERC who are looking the process of developing materials and how the materials are used.

The materials will include professionally-produced videos exemplifying a particular mathematical practice being implemented in a classroom as well as printed dialogues that are designed to help teachers understand the practice and why it is critical for students to acquire that mathematical practice. The exemplars of mathematical practices are being developed based on pilot work and systematic advice from mathematicians, mathematics educators and mathematics teachers in grades 5-10. The design process is iterative and materials are refined based on feedback that is received. Facilitators are being prepared to conduct professional development and materials are being tested by more than 150 teachers in a variety of school districts.

Professional groups such as NCTM and NCSM have called for materials that exemplify the CCSS mathematical practices. They have argued that teachers need to understand how these standards can be achieved in classrooms. IMPS systematic effort to design materials that exemplify the standards and to test not only the materials but also the professional development associated with the materials is responding to the national need. The videos and dialogues will be available through broad dissemination.

Professional Development for Culturally Relevant Teaching and Learning in Pre-K Mathematics

This project is creating and studying a professional development model to support preK teachers in developing culturally and developmentally appropriate practices in counting and early number. The proposed model is targeted at teachers of children in four-year-old kindergarten, and focuses on culturally relevant teaching and learning. The model stresses counting and basic number operations with the intention of exploring the domain as it connects to children's experiences in their homes and communities.

Award Number: 
1019431
Funding Period: 
Wed, 09/01/2010 to Fri, 08/31/2018
Project Evaluator: 
Victoria Jacobs
Full Description: 

Developers and researchers at the University of Wisconsin are creating and studying a professional development model that connects research in counting and early number (CGI), early childhood, and funds of knowledge. The proposed model is targeted at teachers of children in four-year-old kindergarten, and focuses on culturally relevant teaching and learning. The model stresses a specific, circumscribed content domain - counting and basic number operations - with the intention of exploring the domain in depth particularly as it connects to children's experiences in their homes and communities and how it is learned and taught through play.

The project designs, develops, and tests innovative resources and models for teachers to support ongoing professional learning communities. These learning communities are designed to identify and build on the rich mathematical understandings of all pre-K children. The project's specific goals are to instantiate a reciprocal "funds of knowledge" framework for (a) accessing children's out-of-school experiences in order to provide instruction that is equitable and culturally relevant and (b) developing culturally effective ways to support families in understanding how to mathematize their children's out-of-school activities. Teachers are observed weekly during the development and evaluation process and student assessments are used to measure students' progress toward meeting project benchmarks and the program's effectiveness in reducing or eliminating the achievement gap.

The outcome is a complete professional development model that includes written and digital materials. The product includes case studies, classroom video, examples of student work, and strategies for responding to students' understandings.

Integrating Computing Across the Curriculum (ICAC): Incorporating Technology into STEM Education Using XO Laptops

This project builds and tests applications tied to the school curriculum that integrate the sciences with mathematics, computational thinking, reading and writing in elementary schools. The investigative core of the project is to determine how to best integrate computing across the curriculum in such a way as to support STEM learning and lead more urban children to STEM career paths.

Project Email: 
Award Number: 
1404467
Funding Period: 
Sat, 08/01/2009 to Sun, 07/31/2011
Project Evaluator: 
Leslie Cooksy - Univ. of Delaware
Full Description: 

Computer access has opened an exciting new dimension for STEM education; however, if computers in the classroom are to realize their full potential as a tool for advancing STEM education, methods must be developed to allow them to serve as a bridge across the STEM disciplines. The goal of this 60-month multi-method, multi-disciplinary ICAC project is to develop and test a program to increase the number of students in the STEM pipeline by providing teachers and students with curricular training and skills to enhance STEM education in elementary schools. ICAC will be implemented in an urban and predominantly African American school system, since these schools traditionally lag behind in filling the STEM pipeline. Specifically, ICAC will increase computer proficiency (e.g., general usage and programming), science, and mathematics skills of teachers and 4th and 5th grade students, and inform parents about the opportunities available in STEM-centered careers for their children.

The Specific Aims of ICAC are to:

SA1. Conduct a formative assessment with teachers to determine the optimal intervention to ensure productive school, principal, teacher, and student participation.

SA2. Implement a structured intervention aimed at (1) teachers, (2) students, and (3) families that will enhance the students’ understanding of STEM fundamentals by incorporating laptops into an inquiry-based educational process.

SA3. Assess the effects of ICAC on:

a. Student STEM  engagement and performance.

b. Teacher and student computing specific confidence and utilization.

c. Student interest in technology and STEM careers.

d. Parents’ attitudes toward STEM careers and use of computers.

To enable us to complete the specific aims noted above, we have conducted a variety of project activities in Years 1-3. These include:

  1. Classroom observations at the two Year 1 pilot schools
  2. Project scaling to 6 schools in Year 2 and 10 schools in Year 3
  3. Semi-structured school administrator interviews in schools
  4. Professional development sessions for teachers
  5. Drafting of curriculum modules to be used in summer teacher institutes and for dissemination
  6. In-class demonstration of curriculum modules
  7. Scratch festivals each May
  8. Summer teacher institutes
  9. Student summer camps
  10. Surveying of teachers in summer institutes
  11. Surveying of teachers and students at the beginning and end of the school year
  12. Showcase event at end of student workshops

The specific ICAC activities for Years 2-5 include:

  • Professional development sessions (twice monthly for teachers), to integrate the ‘best practices’ from the program.
  • Working groups led by a grade-specific lead teacher. The lead teacher for each grade in each school will identify areas where assistance is needed and will gather the grade-specific cohort of teachers at their school once every two weeks for a meeting to discuss the progress made in addition to challenges to or successes in curricula development.  
  • ICAC staff and prior trained teachers will visit each class monthly during the year to assist the teachers and to evaluate specific challenges and opportunities for the use of XOs in that classroom.  
  • In class sessions at least once per month (most likely more often given feedback from Teacher Summer Institutes) to demonstrate lesson plans and assist teachers as they implement lesson plans.
  • ICAC staff will also hold a joint meeting of administrators of all target schools each year to assess program progress and challenges. 
  • Teacher Summer Institutes – scaled-up to teachers from the new schools each summer to provide training in how to incorporate computing into their curriculum.
  • Administrator sessions during the Teacher Summer Institutes; designed to provide insight into how the laptops can facilitate the education and comprehension of their students in all areas of the curriculum, discuss flexible models for physical classroom organization to facilitate student learning, and discussions related to how to optimize the use of computing to enhance STEM curricula in their schools.  Student Summer Computing Camps – designed to teach students computing concepts, make computing fun, and enhance their interest in STEM careers.  
  • ICAC will sponsor a yearly showcase event in Years 2-5 that provides opportunities for parents to learn more about technology skills their children are learning (e.g., career options in STEM areas, overview of ICAC, and summary of student projects). At this event, a yearly citywide competition among students also will be held that is an expanded version of the weeklong showcase event during the student summer camps.
  • Surveying of students twice a year in intervention schools.
  • Surveying of teachers at Summer Institutes and then at the end of the academic year.
  • Coding and entry of survey data; coding of interview and observational data.
  • Data analysis to examine the specific aims (SA) noted above:
    • The impact of ICAC on teacher computing confidence and utilization (SA 3.b).
    • Assess the effects of (1) teacher XO training on student computing confidence and utilization (SA 3.b), (2) training on changes in interest in STEM careers (SA 3.c), and (3) XO training on student engagement (SA 3.a).
    • A quasi-experimental comparison of intervention and non-intervention schools to assess intervention effects on student achievement (SA 3.a).
    • Survey of parents attending the yearly ICAC showcase to assess effects on parental attitudes toward STEM careers and computing (SA 3.d).

The proposed research has the potential for broad impact by leveraging technology in BCS to influence over 8,000 students in the Birmingham area. By targeting 4th and 5th grade students, we expect to impact STEM engagement and preparedness of students before they move into a critical educational and career decision-making process. Further, by bolstering student computer and STEM knowledge, ICAC will impart highly marketable skills that prepare them for the 81% of new jobs that are projected to be in computing and engineering in coming years (as predicted by the US Bureau of Labor Statistics).3 Through its formative and summative assessment, ICAC will offer intellectual merit by providing teachers throughout the US with insights into how computers can be used to integrate the elementary STEM curriculum. ICAC will develop a model for using computers to enhance STEM education across the curriculum while instilling a culture among BCS schools where computing is viewed as a tool for learning.

(Previously listed under Award # 0918216)

Pages

Subscribe to Effectiveness