Disciplinary Content Knowledge

Investigating How to Enhance Scientific Argumentation through Automated Feedback in the Context of Two High School Earth Science Curriculum Units

This project responds to the need for technology-enhanced assessments that promote the critical practice of scientific argumentation--making and explaining a claim from evidence about a scientific question and critically evaluating sources of uncertainty in the claim. It will investigate how to enhance this practice through automated scoring and immediate feedback in the context of two high school curriculum units--climate change and fresh-water availability--in schools with diverse student populations. 

Lead Organization(s): 
Award Number: 
1418019
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

With the current emphasis on learning science by actively engaging in the practices of science, and the call for integration of instruction and assessment; new resources, models, and technologies are being developed to improve K-12 science learning. Student assessment has become a nationwide educational priority due, in part, to the need for relevant and timely data that inform teachers, administrators, researchers, and the public about how all students perform and think while learning science. This project responds to the need for technology-enhanced assessments that promote the critical practice of scientific argumentation--making and explaining a claim from evidence about a scientific question and critically evaluating sources of uncertainty in the claim. It will investigate how to enhance this practice through automated scoring and immediate feedback in the context of two high school curriculum units--climate change and fresh-water availability--in schools with diverse student populations. The project will apply advanced automated scoring tools to students' written scientific arguments, provide individual students with customized feedback, and teachers with class-level information to assist them with improving scientific argumentation. The key outcome of this effort will be a technology-supported assessment model of how to advance the understanding of argumentation, and the use of multi-level feedback as a component of effective teaching and learning. The project will strengthen the program's current set of funded activities on assessment, focusing these efforts on students' argumentation as a complex science practice.

This design and development research targets high school students (n=1,940) and teachers (n=22) in up to 10 states over four years. The research questions are: (1) To what extent can automated scoring tools, such as c-rater and c-rater-ML, diagnose students' explanations and uncertainty articulations as compared to human diagnosis?; (2) How should feedback be designed and delivered to help students improve scientific argumentation?; (3) How do teachers use and interact with class-level automated scores and feedback to support students' scientific argumentation with real-data and models?; and (4) How do students perceive their overall experience with the automated scores and immediate feedback when learning core ideas in climate change and fresh-water availability topics through scientific argumentation enhanced with modeling? In Years 1 and 2, plans are to conduct feasibility studies to build automated scoring models and design feedback for previously tested assessments for the two curriculum units. In Year 3, the project will implement design studies in order to identify effective feedback through random assignment. In Year 4, a pilot study will investigate if effective feedback should be offered with or without scores. The project will employ a mixed-methods approach. Data-gathering strategies will include classroom observations; screencast and log data of teachers' and students' interaction with automated feedback; teachers' and students' surveys with selected- and open-ended questions; and in-depth interviews with teachers and students. All constructed-response explanations and uncertainty items will be scored using automated scoring engines with fine-grained rubrics. Data analysis strategies will include multiple criteria to evaluate the quality of automated scores; descriptive statistical abalyses; analysis of variance to investigate differences in outcomes from the designed studies' pre/posttests and embedded assessments; analysis of covariance to investigate student learning trajectories; two-level hierarchical linear modeling to study the clustering of students within a class; and analysis of screencasts and log data.

GRIDS: Graphing Research on Inquiry with Data in Science

The Graphing Research on Inquiry with Data in Science (GRIDS) project will investigate strategies to improve middle school students' science learning by focusing on student ability to interpret and use graphs. GRIDS will undertake a comprehensive program to address the need for improved graph comprehension. The project will create, study, and disseminate technology-based assessments, technologies that aid graph interpretation, instructional designs, professional development, and learning materials.

Award Number: 
1418423
Funding Period: 
Mon, 09/01/2014 to Sat, 08/31/2019
Full Description: 

The Graphing Research on Inquiry with Data in Science (GRIDS) project is a four-year full design and development proposal, addressing the learning strand, submitted to the DR K-12 program at the NSF. GRIDS will investigate strategies to improve middle school students' science learning by focusing on student ability to interpret and use graphs. In middle school math, students typically graph only linear functions and rarely encounter features used in science, such as units, scientific notation, non-integer values, noise, cycles, and exponentials. Science teachers rarely teach about the graph features needed in science, so students are left to learn science without recourse to what is inarguably a key tool in learning and doing science. GRIDS will undertake a comprehensive program to address the need for improved graph comprehension. The project will create, study, and disseminate technology-based assessments, technologies that aid graph interpretation, instructional designs, professional development, and learning materials.

GRIDS will start by developing the GRIDS Graphing Inventory (GGI), an online, research-based measure of graphing skills that are relevant to middle school science. The project will address gaps revealed by the GGI by designing instructional activities that feature powerful digital technologies including automated guidance based on analysis of student generated graphs and student writing about graphs. These materials will be tested in classroom comparison studies using the GGI to assess both annual and longitudinal progress. Approximately 30 teachers selected from 10 public middle schools will participate in the project, along with approximately 4,000 students in their classrooms. A series of design studies will be conducted to create and test ten units of study and associated assessments, and a minimum of 30 comparison studies will be conducted to optimize instructional strategies. The comparison studies will include a minimum of 5 experiments per term, each with 6 teachers and their 600-800 students. The project will develop supports for teachers to guide students to use graphs and science knowledge to deepen understanding, and to develop agency and identity as science learners.

From Elementary Generalist to Mathematics Specialist: Examining Teacher Practice and Student Outcomes in Departmental and Self-Contained Models

This research investigates student mathematics learning outcomes at the elementary level in relation to teacher expertise (elementary teachers with math specialist certification versus generally prepared elementary teachers) and school organization (departmentalized versus self-contained mathematics classrooms). Findings will provide evidence of the impact of content-specific teacher expertise and a departmentalized school organizational model that offers students access to well-qualified teachers of mathematics with no additional staffing costs.

Lead Organization(s): 
Award Number: 
1414438
Funding Period: 
Fri, 08/01/2014 to Tue, 07/31/2018
Full Description: 

This research investigates student mathematics learning outcomes at the elementary level in relation to teacher expertise (elementary teachers with math specialist certification versus generally prepared elementary teachers) and school organization (departmentalized versus self-contained mathematics classrooms). University of Missouri researchers will organize and facilitate the research in multiple Missouri public and private school sites. Findings will provide evidence of the impact of content-specific teacher expertise and a departmentalized school organizational model that offers students access to well-qualified teachers of mathematics with no additional staffing costs. To investigate the impact of teacher expertise and school organization on student learning experimental, quasi-experimental, and qualitative designs are employed. Specifically, 80 teachers who have earned state certification as Elementary Mathematics Specialists (hereafter, "EMS teachers") will be selected to participate in the study. Employing a randomized experimental design, half of the EMS teachers will be assigned to teach in a departmental model (Condition 1) in their school, teaching two or more sections of grade 4 mathematics. The other half will remain in generalist (hereafter, self-contained) positions (Condition 2) in their school, teaching all regular subjects to a single class of students. A comparison group of 40 non-EMS teachers with self-contained teaching assignments (Condition 3) will be selected from the same schools in Condition 2. The Smarter Balanced assessment will provide a baseline measure of students' prior achievement in grade 3 and also a measure of the mathematics achievement of grade 4 students taught by the 120 teachers in the study. The project team will analyze student-level mathematics scores linked with specific teachers.

Improving student achievement in mathematics at the elementary level is particularly challenging due to the way elementary students are generally organized for instruction (e.g., one teacher responsible for teaching all subjects to 25-30 students). Because elementary teachers must be knowledgeable about many content areas, they rarely study mathematics in depth, even though there is a growing body of evidence showing the impact of specialized mathematical knowledge for teaching on student learning outcomes. This study carefully researches an alternative model, where elementary teachers with special training in mathematics teaching and learning are assigned more than one single class of students for mathematics instruction. Findings from the research will provide evidence about the impact of both certification as an elementary mathematics specialist and teaching in a departmental assignment on student learning. The results will help school and district leaders better plan for assignment of staff to provide all students with access to high quality mathematics instruction.

Focus on Energy: Preparing Elementary Teachers to Meet the NGSS Challenge (Collaborative Research: Seeley)

This project will develop and investigate the opportunities and limitations of Focus on Energy, a professional development (PD) system for elementary teachers (grades 3-5). The PD will contain: resources that will help teachers to interpret, evaluate and cultivate students' ideas about energy; classroom activities to help them to identify, track and represent energy forms and flows; and supports to help them in engaging students in these activities.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1418211
Funding Period: 
Mon, 09/01/2014 to Fri, 08/31/2018
Full Description: 

The Next Generation Science Standards (NGSS) identify an ambitious progression for learning energy, beginning in elementary school. To help the nation's teachers address this challenge, this project will develop and investigate the opportunities and limitations of Focus on Energy, a professional development (PD) system for elementary teachers (grades 3-5). The PD will contain: resources that will help teachers to interpret, evaluate and cultivate students' ideas about energy; classroom activities to help them to identify, track and represent energy forms and flows; and supports to help them in engaging students in these activities. Teachers will receive the science and pedagogical content knowledge they need to teach about energy in a crosscutting way across all their science curricula; students will be intellectually engaged in the practice of developing, testing, and revising a model of energy they can use to describe phenomena both in school and in their everyday lives; and formative assessment will guide the moment-by-moment advancement of students' ideas about energy.

This project will develop and test a scalable model of PD that will enhance the ability of in-service early elementary teachers to help students learn energy concepts by coordinating formative assessment, face-to-face and web-based PD activities. Researchers will develop and iteratively refine tools to assess both teacher and student energy reasoning strategies. The goals of the project include (1) teachers' increased facility with, and disciplined application of, representations and energy reasoning to make sense of everyday phenomena in terms of energy; (2) teachers' increased ability to interpret student representations and ideas about energy to make instructional decisions; and (3) students' improved use of representations and energy reasoning to develop and refine models that describe energy forms and flows associated with everyday phenomena. The web-based product will contain: a set of formative assessments to help teachers to interpret student ideas about energy based on the Facets model; a series of classroom tested activities to introduce the Energy Tracking Lens (method to explore energy concept using multiple representations); and videos of classroom exemplars as well as scientists thinking out loud while using the Energy Tracking Lens. The project will refine the existing PD and build a system that supports online implementation by constructing a facilitator's guide so that the online community can run with one facilitator.

EarSketch: An Authentic, Studio-based STEAM Approach to High School Computing Education

This project will study the influence on positive student achievement and engagement (particularly among populations traditionally under-represented in computer science) of an intervention that integrates a computational music remixing tool -EarSketch- with the Computer Science Principles, a view of computing literacy that is emerging as a new standard for Advanced Placement and other high school computer science courses.

Award Number: 
1417835
Funding Period: 
Fri, 08/01/2014 to Tue, 07/31/2018
Project Evaluator: 
Mary Moriarity
Full Description: 

This project will study the influence on positive student achievement and engagement (particularly among populations traditionally under-represented in computer science) of an intervention that integrates a computational music remixing tool -EarSketch- with the Computer Science Principles, a view of computing literacy that is emerging as a new standard for Advanced Placement and other high school computer science courses. The project is grounded on the premise that EarSketch, a STEM + Art (STEAM) learning environment, embodies authenticity (i.e., its cultural and industry relevance in both arts and STEM domains), along with a context that facilitates communication and collaboration among students (i.e., through a studio-based learning approach). These elements are critical to achieving successful outcomes across diverse student populations. Using agent-based modeling, the research team will investigate what factors enhance or impede implementation of authentic STEAM tools in different school settings.

The researchers will be engaged in a multi-stage process to develop: a) an implementation-ready, web-based EarSketch learning environment that integrates programming, digital audio workstation, curriculum, audio loop library, and social sharing features, along with studio-based learning functionality to support student presentation, critique, discussion, and collaboration; and b) an online professional learning course for teachers adopting EarSketch in Computer Science Principles courses. Using these resources, the team will conduct a quasi-experimental study of EarSketch in Computer Science Principles high school courses across the state of Georgia; measure student learning and engagement across multiple demographic categories; and determine to what extent an EarSketch-based CS Principles course promotes student achievement and engagement across different student populations. The project will include measures of student performance, creativity, collaboration, and communication in student programming tasks to determine the extent to which studio-based learning in EarSketch promotes success in these important areas. An agent-based modeling framework in multiple school settings will be developed to determine what factors enhance or impede implementation of EarSketch under conditions of routine practice.

Driven to Discover: Citizen Science Inspires Classroom Investigation

This project utilizes existing citizen science programs as springboards for professional development for teachers during an intensive summer workshop. The project curriculum helps teachers use student participation in citizen science to engage them in the full complement of science practices; from asking questions, to conducting independent research, to sharing findings.

Award Number: 
1417777
Funding Period: 
Wed, 10/01/2014 to Sun, 09/30/2018
Full Description: 

Citizen science refers to partnerships between volunteers and scientists that answer real world questions. The target audiences in this project are middle and high school teachers and their students in a broad range of settings: two urban districts, an inner-ring suburb, and three rural districts. The project utilizes existing citizen science programs as springboards for professional development for teachers during an intensive summer workshop. The project curriculum helps teachers use student participation in citizen science to engage them in the full complement of science practices; from asking questions, to conducting independent research, to sharing findings. Through district professional learning communities (PLCs), teachers work with district and project staff to support and demonstrate project implementation. As students and their teachers engage in project activities, the project team is addressing two key research questions: 1) What is the nature of instructional practices that promote student engagement in the process of science?, and 2) How does this engagement influence student learning, with special attention to the benefits of engaging in research presentations in public, high profile venues? Key contributions of the project are stronger connections between a) ecology-based citizen science programs, STEM curriculum, and students' lives and b) science learning and disciplinary literacy in reading, writing and math.

Research design and analysis are focused on understanding how professional development that involves citizen science and independent investigations influences teachers' classroom practices and student learning. The research utilizes existing instruments to investigate teachers' classroom practices, and student engagement and cognitive activity: the Collaboratives for Excellence in Teacher Preparation and Classroom Observation Protocol, and Inquiring into Science Instruction Observation Protocol. These instruments are used in classroom observations of a stratified sample of classes whose students represent the diversity of the participating districts. Curriculum resources for each citizen science topic, cross-referenced to disciplinary content and practices of the NGSS, include 1) a bibliography (books, web links, relevant research articles); 2) lesson plans and student science journals addressing relevant science content and background on the project; and 3) short videos that help teachers introduce the projects and anchor a digital library to facilitate dissemination. Impacts beyond both the timeframe of the project and the approximately 160 teachers who will participate are supported by curriculum units that address NGSS life science topics, and wide dissemination of these materials in a variety of venues. The evaluation focuses on outcomes of and satisfaction with the summer workshop, classroom incorporation, PLCs, and student learning. It provides formative and summative findings based on qualitative and quantitative instruments, which, like those used for the research, have well-documented reliability and validity. These include the Science Teaching Efficacy Belief Instrument to assess teacher beliefs; the Reformed Teaching Observation Protocol to assess teacher practices; the Standards Assessment Inventory to assess PLC quality; and the Scientific Attitude Inventory to assess student attitudes towards science. Project deliverables include 1) curriculum resources that will support engagement in five existing citizen science projects that incorporate standards-based science content; 2) venues for student research presentations that can be duplicated in other settings; and 3) a compilation of teacher-adapted primary scientific research articles that will provide a model for promoting disciplinary literacy. The project engages 40 teachers per year and their students.

Changing Culture in Robotics Classroom (Collaborative Research: Shoop)

Computational and algorithmic thinking are new basic skills for the 21st century. Unfortunately few K-12 schools in the United States offer significant courses that address learning these skills. However many schools do offer robotics courses. These courses can incorporate computational thinking instruction but frequently do not. This research project aims to address this problem by developing a comprehensive set of resources designed to address teacher preparation, course content, and access to resources.

Lead Organization(s): 
Partner Organization(s): 
Award Number: 
1418199
Funding Period: 
Mon, 09/01/2014 to Thu, 08/31/2017
Full Description: 

Computational and algorithmic thinking are new basic skills for the 21st century. Unfortunately few K-12 schools in the United States offer significant courses that address learning these skills. However many schools do offer robotics courses. These courses can incorporate computational thinking instruction but frequently do not. This research project aims to address this problem by developing a comprehensive set of resources designed to address teacher preparation, course content, and access to resources. This project builds upon a ten year collaboration between Carnegie Mellon's Robotics Academy and the University of Pittsburgh's Learning Research and Development Center that studied how teachers implement robotics education in their classrooms and developed curricula that led to significant learning gains. This project will address the following three questions:

1.What kinds of resources are useful for motivating and preparing teachers to teach computational thinking and for students to learn computational thinking?

2.Where do teachers struggle most in teaching computational thinking principles and what kinds of supports are needed to address these weaknesses?

3.Can virtual environments be used to significantly increase access to computational thinking principles?

The project will augment traditional robotics classrooms and competitions with Robot Virtual World (RVW) that will scaffold student access to higher-order problems. These virtual robots look just like real-world robots and will be programmed using identical tools but have zero mechanical error. Because dealing with sensor, mechanical, and actuator error adds significant noise to the feedback students' receive when programming traditional robots (thus decreasing the learning of computational principles), the use of virtual robots will increase the learning of robot planning tasks which increases learning of computational thinking principles. The use of RVW will allow the development of new Model-Eliciting Activities using new virtual robotics challenges that reward creativity, abstraction, algorithms, and higher level programming concepts to solve them. New curriculum will be developed for the advanced concepts to be incorporated into existing curriculum materials. The curriculum and learning strategies will be implemented in the classroom following teacher professional development focusing on computational thinking principles. The opportunities for incorporating computationally thinking principles in the RVW challenges will be assessed using detailed task analyses. Additionally regression analyses of log-files will be done to determine where students have difficulties. Observations of classrooms, surveys of students and teachers, and think-alouds will be used to assess the effectiveness of the curricula in addition to pre-and post- tests to determine student learning outcomes.

Centers for Learning and Teaching: Research to Identify Changes in Mathematics Education Doctoral Preparation and the Production of New Doctorates

This project will research the programmatic changes that resulted from the NSF investment in Centers for Learning and Teaching of Mathematics (CLT) at the 31 participating institutions. It will provide information on the core elements of doctoral preparation in mathematics education at the institutions and ways in which participation in the CLTs has changed their programs.

Lead Organization(s): 
Award Number: 
1434442
Funding Period: 
Fri, 08/01/2014 to Tue, 07/31/2018
Full Description: 

The quality of the mathematical education provided to teachers and ultimately to their students depends on the quality of teacher educators at the colleges and universities. For several decades, there has been a shortage of well-prepared mathematics teacher educators. Doctoral programs in mathematics education are the primary ways that these teacher educators learn the content and methods that they need to prepare teachers, but the quality of these programs varies and the number of qualified graduates has been insufficient to meet the demand.

This project will research the programmatic changes that resulted from the NSF investment in Centers for Learning and Teaching of Mathematics (CLT) at the 31 participating institutions. It will provide information on the core elements of doctoral preparation in mathematics education at the institutions and ways in which participation in the CLTs has changed their programs. It will also gather data on the number of doctorates in mathematics education from the CLT institutions prior to the establishment of the CLT and after their CLT ended. A comparison group of Doctoral granting institutions will be studied over the same time frame to determine the number of doctoral students graduated during similar time frames as the CLTs. Follow-up data from graduates of the CLTs will be gathered to identify programmatic strengths and weaknesses as graduates will be asked to reflect on how their doctoral preparation aligned with their current career path. The research questions are: What were the effects of CLTs on the production of new doctorates in mathematics education? What changes were made to doctoral programs in mathematics education by the CLT institutions? How well prepared were the CLT graduates for various career paths?

Multimedia Engineering Notebook Tools to Support Engineering Discourse in Urban Elementary School Classrooms (Collaborative Research: Paugh)

This collaborative, exploratory, learning strand project focuses on improving reflective decision-making among elementary school students during the planning and re-design activities of the engineering design process. Five teacher researchers in three elementary schools provide the classroom laboratories for the study. Specified units from Engineering is Elementary, a well-studied curriculum, provide the engineering content.

Award Number: 
1316762
Funding Period: 
Thu, 08/01/2013 to Sun, 07/31/2016
Full Description: 

This collaborative, exploratory, learning strand project focuses on improving reflective decision-making among elementary school students during the planning and re-design activities of the engineering design process. Five teacher researchers in three elementary schools provide the classroom laboratories for the study. Specified units from Engineering is Elementary, a well-studied curriculum, provide the engineering content. In year one, the qualitative research observes student discourse as students develop designs. Based on the results, a paper engineering note book with prompts is designed for use in year two while a digital notebook is developed. In year three, the students use the digital notebook to develop their designs and redesigns.

The research identifies patterns of language that contribute to the reflective discourse and determines how the paper and electronic versions of the notebook improve the discourse. An advisory committee provides advice and evaluation. The notebooks are described in conference proceedings and made available online.

This work synthesizes what is known about the use of the notebooks in science and engineering education at the elementary school and investigates how to improve their use through digital media.

Integrating Quality Talk Professional Development to Enhance Professional Vision and Leadership for STEM Teachers in High-Need Schools

This project expands and augments a currently-funded NSF Noyce Track II teacher recruitment and retention grant with Quality Talk (QT), an innovative, scalable teacher-facilitated discourse model. Over the course of four years, the work will address critical needs in physics and chemistry education in 10th through 12th grade classrooms by strengthening the capacity of participating teachers to design and implement lessons that support effective dialogic interactions.

Award Number: 
1316347
Funding Period: 
Mon, 07/15/2013 to Fri, 06/30/2017
Full Description: 

This project expands and augments a currently-funded NSF Noyce Track II teacher recruitment and retention grant with Quality Talk (QT), an innovative, scalable teacher-facilitated discourse model. It is hypothesized that the QT model will enhance pre- and in-service secondary teachers' development of professional vision and leadership skills necessary for 21st century STEM education. Over the course of four years, the work will address critical needs in physics and chemistry education in 10th through 12th grade classrooms in five of Georgia's high-need school districts by strengthening the capacity of participating teachers to design and implement lessons that support effective dialogic interactions. As a result of such interactions, students' scientific literacy will be enhanced, including their ability to participate in content-rich discourse (i.e., QT) through effective disciplinary critical-analytic thinking and epistemic cognition. The contributions of this project, beyond the tangible benefits for teacher and student participants, include the development, refinement, and dissemination of an effective QT intervention and professional developmental framework that the entire science education community can use to promote scientific literacy and understanding.

The project goals are being achieved through a series of three studies employing complementary methods and data sources, and a focus upon dissemination of the model in the final project year. The first two years of the project focus on developing and refining the curricular and intervention efficacy materials using design-based research methods. In Year 3, the project engages in a quasi-experimental study of the refined QT model, followed by further refinements before disseminating the materials both within Georgia and throughout the national science education community in Year 4. Quantitative measures of teacher and student discourse and knowledge, as well as video-coding and qualitative investigations of intervention efficacy, are being analyzed using multiple methods. In collaboration with, but independent from project staff and stakeholders, the participatory and responsive evaluation utilizes a variety of qualitative and quantitative methods to conduct formative and summative evaluation.

Over the course of four years, the project will involve the participation of approximately 32 teachers in Georgia whose students include substantive percentages from populations underrepresented in the STEM fields. In addition to advancing their own students' scientific literacy, these participating teachers receive professional development on how to train other teachers, outside of the project, in using QT to promote scientific literacy. Further, the project will conduct a QT Summit for educational stakeholders and non-participant teachers to disseminate the intervention and professional development model. Finally, the project team will disseminate the findings widely to applied and scholarly communities through a website with materials and PD information (http://www.qualitytalk.org), professional journals, conferences, and NSF's DRK-12 Resource Network. This project, with its focus on teacher leadership and the pedagogical content knowledge necessary to use discourse to promote student science literacy, significantly advances the nation's goals of producing critical consumers and producers of scientific knowledge.

Pages

Subscribe to Disciplinary Content Knowledge