Mixed Methods

Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum (Collaborative Research: Henderson)

The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Lead Organization(s): 
Award Number: 
1621496
Funding Period: 
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description: 

This is an early-stage design and development collaborative study submitted to the assessment strand of the Discovery Research PreK-12 (DRK-12) program, in response to Program Solicitation NSF 15-592. The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. To achieve this purpose, the project will examine the validity of a new technology-based formative assessment tool for classroom argumentation--"Diagnosing the Argumentation Levels of Groups" (DiALoG)--for which psychometric validation work has been conducted in a laboratory setting. The DiALoG assessment tool allows teachers to document classroom talk and display scores across multiple dimensions--both intrapersonal and interpersonal--for formative assessment purposes. The project will work with 6th-8th grade science teachers to monitor and support argumentation through real-time formative assessment data generated by the DiALoG instrument. DiALoG will be used in conjunction with "Amplify Science", a Lawrence Hall of Science-developed curriculum that incorporates the science practice of engaging in argument from evidence, and a suite of newly developed Responsive Mini-Lessons (RMLs), which consist of 20-30 minute instructional strategies designed to assist teachers to provide feedback to students' thinking and follow-up to argumentation episodes that the DiALoG tool identifies in need of further support. The study will allow the refinement and expansion of DiALoG and evaluation of its impact on teacher pedagogical content knowledge and formative assessment practices in widespread classroom use.

The project will address two specific research questions: (1) How can DiALoG be refined to provide a formative assessment tool for oral argumentation that is reliable, practical, and useful in middle school classrooms?; and (2) How does the use of DiALoG affect teacher formative assessment practices around evidence-based argumentation, when implementing science units designed to support oral argumentation? In order to answer these questions, the project will conduct a randomized control trial with 100 teachers: 50 will teach argumentation-focused curriculum with DiALoG, 50 will teach the same curriculum without DiALoG. Both control and treatment teachers will receive all digital and physical materials needed to teach three Amplify Science curriculum units. Treatment teachers will be provided also with the most recent version of DiALoG, including the linked RMLs, as well as support materials for using DiALoG with the Amplify curriculum. A subgroup of focus teachers (5 from the treatment group, and 5 from the control group) will be the subject of additional data collection and analysis. Three focus lessons, in which students are engaging in small-group or whole-class oral argumentation, will be selected from each of the three Amplify Science curricular units. Teacher measures for the randomized control trial will include validated instruments, such as (a) a pre- and post-assessment of teacher pedagogical content knowledge; (b) post-lesson and post-unit surveys in which teachers will self-report on their formative assessment practices; and (c) video recordings of selected lessons in the focus classrooms. In order to observe potential differences in formative assessment practices between treatment and control, protocols will be used to analyze the video recordings of focus classrooms, including (a) Reformed Teaching Observation Protocol; (b) Assessment of Scientific Argumentation inside the Classroom; and (c) Formative Assessment for Teachers and Students. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.

Systemic Transformation of Inquiry Learning Environments for STEM (STILE 2.0)

The project is a four-year, early-stage design and development project aimed to refine a state-of-the-art professional development model to prepare K-8 teachers and instructional leaders in urban schools to facilitate and support successful K-8 STEM Education. The project will specifically explore which components of the program promote teacher change, which aspects of the program support structural changes for STEM teaching in schools, and what holds promise for interdisciplinary STEM teacher development.

Lead Organization(s): 
Award Number: 
1621387
Funding Period: 
Thu, 09/01/2016 to Mon, 08/31/2020
Full Description: 

The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

The project at The Center for Technology and School Change (CTSC) at Teachers College, Columbia University, is a four-year, early-stage design and development project aimed to refine a state-of-the-art professional development model to prepare K-8 teachers and instructional leaders in urban schools to facilitate and support successful K-8 STEM Education. This project will explore the most effective features for preparing teachers to design and implement authentic STEM learning experiences in twelve high need elementary and middle urban schools across New York City and Yorkers. The project will specifically explore which components of the program promote teacher change, which aspects of the program support structural changes for STEM teaching in schools, and what holds promise for interdisciplinary STEM teacher development.

Participants in this project will design and implement transdisciplinary STEM projects and learn to develop and support STEM learning environments for their schools. As part of this overall process, researchers will refine a situated professional development curriculum, including a suite of digital case studies that will assist schools. The project will: 1) build a vision for trans-disciplinary STEM schooling; 2) design and implement STEM learning experiences; and 3) take capacity-building steps to sustain STEM practices. A mixed method design approach will be used to explore both the implementation of the project and the effect of implementation on participants.

CAREER: Making Science Visible: Using Visualization Technology to Support Linguistically Diverse Middle School Students' Learning in Physical and Life Sciences

Award Number: 
1552114
Funding Period: 
Wed, 06/01/2016 to Mon, 05/31/2021
Full Description: 

The growing diversity in public schools requires science educators to address the specific needs of English language learners (ELLs), students who speak a language other than English at home. Although ELLs are the fastest-growing demographic group in classrooms, many are historically underserved in mainstream science classrooms, particularly those from underrepresented minority groups. The significant increase of ELLs at public schools poses a challenge to science teachers in linguistically diverse classrooms as they try to support and engage all students in learning science. The proposed project will respond to this urgent need by investigating the potential benefits of interactive, dynamic visualization technologies, including simulations, animations, and visual models, in supporting science learning for all middle school students, including ELLs. This project will also identify design principles for developing such technology, develop additional ways to support student learning, and provide new guidelines for effective science teachers' professional development that can assist them to better serve students from diverse language backgrounds. The project has the potential to transform traditional science instruction for all students, including underserved ELLs, and to broaden their participation in science.

In collaboration with eighth grade science teachers from two low-income middle schools in North Carolina, the project will focus on three objectives: (1) develop, test, and refine four open-source, web-based inquiry units featuring dynamic visualizations on energy and matter concepts in physical and life sciences, aligned with the Next Generation Science Standards (NGSS); (2) investigate how dynamic visualizations can engage eighth-grade ELLs and native-English-speaking students in science practices and improve their understanding of energy and matter concepts; and (3) investigate which scaffolding approaches can help maximize ELLs' learning with visualizations. Research questions include: (1) Which kinds of dynamic visualizations (simulations, animations, visual models) lead to the best learning outcomes for all students within the four instructional science units?; (2) Do ELLs benefit more from visualizations (or particular kinds of visualizations) than do native-English-speaking students?; and (3) What kinds of additional scaffolding activities (e.g., critiquing arguments vs. generating arguments) are needed by ELLs in order to achieve the greatest benefit? The project will use design-based research and mixed-methods approaches to accomplish its research objectives and address these questions. Furthermore, it will help science teachers develop effective strategies to support students' learning with visualizations. Products from this project, including four NGSS-aligned web-based inquiry units, the visualizations created for the project, professional development materials, and scaffolding approaches for teachers to use with ELLs, will be freely available through a project website and multiple professional development networks. The PI will collaborate with an advisory board of experts to develop the four instructional units, visualizations, and scaffolds, as well as with the participating teachers to refine these materials in an iterative fashion. Evaluation of the materials and workshops will be provided each year by the advisory board members, and their feedback will be used to improve design and implementation for the next year. The advisory board will also provide summative evaluation of student learning outcomes and will assess the success of the teachers' professional development workshops.

CAREER: Designing Learning Environments to Foster Productive and Powerful Discussions Among Linguistically Diverse Students in Secondary Mathematics

Lead Organization(s): 
Award Number: 
1553708
Funding Period: 
Mon, 02/01/2016 to Sun, 01/31/2021
Full Description: 

The project will design and investigate learning environments in secondary mathematics classrooms focused on meeting the needs of English language learners. An ongoing challenge for mathematics teachers is promoting deep mathematics learning among linguistically diverse groups of students while taking into consideration how students' language background influences their classroom experiences and the mathematical understandings they develop. In response to this challenge, this project will design and develop specialized instructional materials and guidelines for teaching fundamental topics in secondary algebra in linguistically diverse classrooms. The materials will incorporate insights from current research on student learning in mathematics as well as insights from research on the role of language in students' mathematical thinking and learning. A significant contribution of the work will be connecting research on mathematics learning generally with research on the mathematics learning of English language learners. In addition to advancing theoretical understandings, the research will also contribute practical resources and guidance for mathematics teachers who teach English language learners. The Faculty Early Career Development (CAREER) program is a National Science Foundation (NSF)-wide activity that offers awards in support of junior faculty who exemplify the role of teacher-scholars through outstanding research, excellent education, and the integration of education and research within the context of the mission of their organizations.

The project is focused on the design of specialized hypothetical learning trajectories that incorporate considerations for linguistically diverse students. One goal for the specialized trajectories is to foster productive and powerful mathematics discussions about linear and exponential rates in linguistically diverse classrooms. The specialized learning trajectories will include both mathematical and language development learning goals. While this project focuses on concepts related to reasoning with linear and exponential functions, the resulting framework should inform the design of specialized hypothetical learning trajectories in other topic areas. Additionally, the project will add to the field's understanding of how linguistically diverse students develop mathematical understandings of a key conceptual domain. The project uses a design-based research framework gathering classroom-based data, assessment data, and interviews with teachers and students to design and refine the learning trajectories. Consistent with a design-based approach, the project results will include development of theory about linguistically diverse students' mathematics learning and development of guidance and resources for secondary mathematics teachers. This research involves sustained collaboration with secondary mathematics teachers and the impacts will include developing capacity of teachers locally, and propagating the results of this work in professional development activities.


Project Videos

2019 STEM for All Video Showcase

Title: Fostering Math Discussions among English Learners

Presenter(s): William Zahner


Transforming Teaching Through Implementing Inquiry (T2I2)

This project explores the use of cyberinfrastructure to significantly enhance the delivery and quality of professional development for grades 8-12 engineering, technology, and design educators. The goal of the project is to study whether the use of highly interactive cyberinfrastructure increases the educator's teaching competencies and how to effectively teach. Student achievement is measured by comparing state assessments in: the curriculum's technology, engineering, and design assessment, end-of-grade mathematics assessment, and end-of-grade science assessment.

Award Number: 
1156629
Funding Period: 
Mon, 08/01/2011 to Fri, 07/31/2015
Full Description: 

Transforming Teaching Through Implementing Inquiry (T2I2) is a full research and development project that explores the use of cyberinfrastructure to significantly enhance the delivery and quality of professional development (PD) for grades 8-12 engineering, technology, and design educators. The goal of the project is to study whether the use of highly interactive cyberinfrastructure increases this target audience's: 1) understanding of engineering design concepts and ability to effectively teach them 2) understanding of how to address student learning needs 3) ability to manage, monitor, and adjust the learning environment 4) use of self assessment to enhance teaching ability and 5) engagement in a community of practice. These issues are of particular interest because of the limited resources in place to prepare pre-service engineering and CTE teachers, as well as a lack of in-service PD.

The content for the PD is grounded in the materials and processes of two projects reviewed by the National Research Council's (NRC) report review committee: Technology Education: Learning by Design for Middle Schools" and "Engineering by Design for High Schools." By incorporating an object-oriented generic system design (learning objects), the cyberinfrastructure is set to be reusable, adaptable, and scalable. These learning objects allow for customization of the learning experience, whereby learning facilitators or learners themselves can configure the system based on their specific needs. Delivering learning objects in an online framework enables teachers to develop and grow in a network community.

A mixed methods approach is used to determine effects of professional development. Student achievement is measured by comparing each site's state assessments in the following areas: the curriculum's technology, engineering, and design assessment, end-of-grade mathematics assessment, and end-of-grade science assessment. Both formative and summative evaluation strategies inform the development and implementation of the project. As such, the project will advance theory, design, and practice in middle and high school engineering.

Three-Dimensional Teaching and Learning: Rebuilding and Researching an Online Middle School Curriculum

This project will develop an online curriculum-based supported by a teacher professional development (PD) program by rebuilding an existing life science unit of Biological Sciences Curriculum Study (BSCS) Middle School Science. The project is designed to be an exemplar of fully digital Next Generation Science Standards (NGSS) aligned resources for teachers and students, creating an NGSS-aligned learning environment combining disciplinary core ideas with science and engineering practices and cross-cutting concepts.

Lead Organization(s): 
Award Number: 
1502571
Funding Period: 
Tue, 09/01/2015 to Sat, 08/31/2019
Full Description: 

This project was funded by the Discovery Research K-12 (DRK-12) program that seeks to significantly enhance the learning and teaching of science, technology, engineering, and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models, and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. The project, in collaboration with Oregon Public Broadcasting, will develop an online curriculum-based supported by a teacher professional development (PD) program by rebuilding an existing life science unit of Biological Sciences Curriculum Study (BSCS) Middle School Science. The materials will include strategically integrated multimedia elements including animations, interactive learning experiences, and enhanced readings for students, as well as classroom videos for teachers that will help all users gain a deeper understanding of three-dimensional learning. The project is designed to be an exemplar of fully digital Next Generation Science Standards (NGSS) aligned resources for teachers and students, creating an NGSS-aligned learning environment combining disciplinary core ideas with science and engineering practices and cross-cutting concepts. Using the powerful affordances of a digital environment, the project will invigorate and inspire learners and support teachers as only a media-rich environment can do.

The project will develop and research the project innovation, the combination of digital instructional materials for students and online teacher PD using a proven lesson-analysis framework. Although prior research has demonstrated the efficacy of the lesson analysis PD and curriculum elements independently, there has been little investigation of their joint ability to transform teaching and learning. The project will merge research and development in this project by incorporating a complex array of multi-component assessment activities, including classroom-based assessments, in a quasi-experimental study. Assessment activities will be designed using an evidence-centered design process that will involve the careful selection and development of assessment tasks, scoring rubrics, and criteria for scoring based on the performance expectations (PEs) and the best ways to elicit evidence about student proficiency with those PEs. The research, carried out by SRI International, will use multi-component tasks that will support inferences about student learning and advance understanding of how to assess NGSS learning. Project research and resources, which will include a digital, middle school life sciences unit, teacher PD and online digital resources, and related assessment tools, which will be widely disseminated to policy makers, researchers, and practitioners.

Exploring Ways to Enhance Science Learning for English Language Learners Through Improvement in Teacher Self-Efficacy Beliefs

This project will explore the influence of a professional learning community model on preparing preservice and novice science teachers to teach in culturally and linguistically diverse classrooms of English language learners. The project will study the effect of a professional learning community model on teachers' self-efficacy beliefs and practices as it relates to teaching science to this population.

Award Number: 
1551974
Funding Period: 
Tue, 09/15/2015 to Thu, 08/31/2017
Full Description: 

The Discovery Research K-12 program seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics by preK-12 students and teachers, through research and development of innovative resources, models, and tools. This project will explore the influence of a professional learning community model on preparing preservice and novice science teachers to teach in culturally and linguistically diverse classrooms of English language learners. The project will study the effect of a professional learning community model on teachers' self-efficacy beliefs and practices as it relates to teaching science to this population. The need to build teacher capacity is two-fold. First, the project will help the state of Georgia meet the needs of a growing population of English language learners. Second, the project will help preservice and novice teachers prepare more English language learners to obtain the STEM knowledge and skills needed to respond to the increasing demand for a stronger national STEM workforce as well as to fill more than 200,000 STEM-related jobs projected to be available in the state by 2018. To address these major needs, this project will provide opportunities that will empower elementary science teachers to collaborate and to continuously improve science teaching and learning for English language learners. The focus will be on improving instructional practices, strengthening content knowledge, and expanding literacy skills through lesson plan development and modification, lesson implementation, individual reflection using assessment data as evidence, peer debriefing, and collaborative generalization by connecting theory to practice.

A sequential mixed methods design will be used to capture data about the impact of the professional learning community on the self-efficacy beliefs of teachers participating in the project. The PI and graduate students will use Web-based surveys, semi-structured interviews, and classroom observations using a quantitated protocol to collect, analyze, and interpret data on the model being explored. An online statistical software program will be used to compute surveys at various checkpoints over the two-year period while interviews will be coded using computer assisted qualitative data analysis software based on constant comparative analysis. If the project achieve its goals, the outcomes will result in products that could lead to information that might inform change in teacher education programs, interventions for classroom practice, and topics for future research.

Exploring Ways to Transform Teaching Practices to Increase Native Hawaiian Students' Interest in STEM

This project will integrate Native Hawaiian cross-cultural practices to explore ways to help teachers know about and know how to connect resources of students' familiar worlds to their science teaching. This project will transform the ways teachers orient their teaching at the upper elementary and middle grades through professional development courses offered at the University of Hawaii at Manoa.

Lead Organization(s): 
Award Number: 
1551502
Funding Period: 
Tue, 09/01/2015 to Fri, 08/31/2018
Full Description: 

This project will integrate Native Hawaiian cross-cultural practices to explore ways to help teachers know about and know how to connect resources of students' familiar worlds to their science teaching. This research is needed since Native Hawaiians are often stereotyped as poor learners; the available STEM workforce falls short of meeting the demands of STEM employers in the state; and as the largest group of public school enrollees, data show a greater decline in percent of students meeting or exceeding proficiency in science at higher grade levels. This project will address these issues by transforming the ways teachers orient their teaching at the upper elementary and middle grades through professional development courses offered at the University of Hawaii at Manoa.

The professional development model for teachers will be situated in the larger national and global contexts of an increasingly technology oriented, urbanized society with associated marginalization of indigenous people whose traditional ecological knowledge and indigenous languages are often overlooked. Guided by the cultural mental model theory and a mixed methods approach, data will be collected through document analysis, surveys, individual and focus group interviews, and pre-post assessments. This approach will capture initials findings about the influence of the professional development model on teaching and learning in science. The end products from this project will be an improved professional development model that is more sensitive to contexts that promote learning by Native Hawaiian students. It will also produce a survey instrument to assess student interest and engagement in science learning whose teachers will have participated in the professional development model being explored. Both outcomes will potentially be instrumental in changing the way approximately 2000 Native Hawaiian students learn about and become more interested in STEM fields through their natural world.

Developing Integrated Elementary Science, Engineering, and Language Arts Curricula Aligned with Next Generation Science Standards

This project will conduct a study to develop and field-test curricula integrating science, engineering, and language arts at the elementary level which is aligned with the Next Generation Science Standards (NGSS).

Award Number: 
1551143
Funding Period: 
Tue, 09/01/2015 to Thu, 08/31/2017
Full Description: 

The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

Developing Integrated Elementary Science, Engineering, and Language Arts Curricula Aligned with Next Generation Science Standards is an exploratory project to conduct a study to develop and field-test curricula integrating science, engineering, and language arts at the elementary level. Research and Curriculum Development team consisting of master elementary science teachers, university professors including science, engineering, and science teacher education faculty, and a science education post doc or graduate student will engage in developing the Next Generation Science Standards (NGSS) aligned curricula integrating science, engineering, and language arts, and publishing STEM education research. The importance of this project will be the development of curricula integrating science, engineering, and language arts at the elementary level. Lesson plans or teaching activities in the integrated curricula will be written in practitioner article format. In the NGSS the engineering design is raised to the same level as scientific inquiry and included as a vital element of science education. This integrated approach aims to provide three-dimensional learning experience as specified in the NGSS to elementary students while meaningfully integrating engineering, science, reading, and writing through real life engineering design problems. The NGSS aligned curricula that will be developed in this project can also be used in other states that adopted the NGSS.

An Integrated curriculum for grades 1-2 will be developed in year 1. In year 2, the project will develop a curriculum for grades 3-5. Each year, the project will develop and field-test a new curriculum, and provide professional development organized around the integrated curriculum to 20 elementary teachers at the Clark County School District in Las Vegas, Nevada.

PlantingScience: Digging Deeper Together - A Model for Collaborative Teacher/Scientist Professional Development

This project will design, develop, and test a new professional development (PD) model for high school biology teachers that focuses on plant biology, an area of biology that teachers feel less prepared to teach. The new PD model will bring teachers and scientists together, in-person and online, to guide students in conducting authentic science investigations and to reflect on instructional practices and student learning.

Lead Organization(s): 
Award Number: 
1502892
Funding Period: 
Thu, 10/01/2015 to Mon, 09/30/2019
Full Description: 

This project will design, develop, and test a new professional development (PD) model for high school biology teachers that focuses on plant biology, an area of biology that teachers feel less prepared to teach. The new PD model will bring teachers and scientists together, in-person and online, to guide students in conducting authentic science investigations and to reflect on instructional practices and student learning. The project will also develop and test the outcomes of a summer institute for teachers and a website that will support the online mentoring of students and the professional development of teachers. Outcomes of the project will include the development of a facilitation guide for the teacher professional development model, a website to support student mentoring and teacher professional development, a series of resources for teachers and scientists to use in working with students, and empirical evidence of the success of the new professional development model.

This full research and development project will employ a pre-test/post-test control group design to test the efficacy of a professional development model for high school biology teachers. The professional development model is grounded in a theory of action based on the premise that when teachers are engaged with scientists and students in a technology-enabled learning community, students will demonstrate higher levels of achievement than those using more traditional instructional materials and methodologies. The means of post-intervention outcome measures will be compared across treatment and comparison groups in a cluster-randomized trial where teachers will be randomly assigned to treatment groups. The study will recruit a nation-wide sample to ensure that participants represent a wide array of geographic and demographic contexts, with preference given to Title 1 schools. The research questions are: a) To what extent does participation in the Digging Deeper community of teachers and scientists affect teacher knowledge and practices? b) To what extent does participation in the Digging Deeper community of teachers and scientists affect scientists? quality of mentorship and teaching? And c) To what extent does student use of the online program and participation in the learning community with scientist mentors affect student learning? Instruments will be developed or adapted to measure relevant student and teacher knowledge, student motivation, and teacher practices. Computer-mediated discourse analysis will be used over the course of the study to track online interactions among students, teachers, and science mentors.

Pages

Subscribe to Mixed Methods