This project builds and tests applications tied to the school curriculum that integrate the sciences with mathematics, computational thinking, reading and writing in elementary schools. The investigative core of the project is to determine how to best integrate computing across the curriculum in such a way as to support STEM learning and lead more urban children to STEM career paths.
Computer access has opened an exciting new dimension for STEM education; however, if computers in the classroom are to realize their full potential as a tool for advancing STEM education, methods must be developed to allow them to serve as a bridge across the STEM disciplines. The goal of this 60-month multi-method, multi-disciplinary ICAC project is to develop and test a program to increase the number of students in the STEM pipeline by providing teachers and students with curricular training and skills to enhance STEM education in elementary schools. ICAC will be implemented in an urban and predominantly African American school system, since these schools traditionally lag behind in filling the STEM pipeline. Specifically, ICAC will increase computer proficiency (e.g., general usage and programming), science, and mathematics skills of teachers and 4th and 5th grade students, and inform parents about the opportunities available in STEM-centered careers for their children.
The Specific Aims of ICAC are to:
SA1. Conduct a formative assessment with teachers to determine the optimal intervention to ensure productive school, principal, teacher, and student participation.
SA2. Implement a structured intervention aimed at (1) teachers, (2) students, and (3) families that will enhance the students’ understanding of STEM fundamentals by incorporating laptops into an inquiry-based educational process.
SA3. Assess the effects of ICAC on:
a. Student STEM engagement and performance.
b. Teacher and student computing specific confidence and utilization.
c. Student interest in technology and STEM careers.
d. Parents’ attitudes toward STEM careers and use of computers.
To enable us to complete the specific aims noted above, we have conducted a variety of project activities in Years 1-3. These include:
- Classroom observations at the two Year 1 pilot schools
- Project scaling to 6 schools in Year 2 and 10 schools in Year 3
- Semi-structured school administrator interviews in schools
- Professional development sessions for teachers
- Drafting of curriculum modules to be used in summer teacher institutes and for dissemination
- In-class demonstration of curriculum modules
- Scratch festivals each May
- Summer teacher institutes
- Student summer camps
- Surveying of teachers in summer institutes
- Surveying of teachers and students at the beginning and end of the school year
- Showcase event at end of student workshops
The specific ICAC activities for Years 2-5 include:
- Professional development sessions (twice monthly for teachers), to integrate the ‘best practices’ from the program.
- Working groups led by a grade-specific lead teacher. The lead teacher for each grade in each school will identify areas where assistance is needed and will gather the grade-specific cohort of teachers at their school once every two weeks for a meeting to discuss the progress made in addition to challenges to or successes in curricula development.
- ICAC staff and prior trained teachers will visit each class monthly during the year to assist the teachers and to evaluate specific challenges and opportunities for the use of XOs in that classroom.
- In class sessions at least once per month (most likely more often given feedback from Teacher Summer Institutes) to demonstrate lesson plans and assist teachers as they implement lesson plans.
- ICAC staff will also hold a joint meeting of administrators of all target schools each year to assess program progress and challenges.
- Teacher Summer Institutes – scaled-up to teachers from the new schools each summer to provide training in how to incorporate computing into their curriculum.
- Administrator sessions during the Teacher Summer Institutes; designed to provide insight into how the laptops can facilitate the education and comprehension of their students in all areas of the curriculum, discuss flexible models for physical classroom organization to facilitate student learning, and discussions related to how to optimize the use of computing to enhance STEM curricula in their schools. Student Summer Computing Camps – designed to teach students computing concepts, make computing fun, and enhance their interest in STEM careers.
- ICAC will sponsor a yearly showcase event in Years 2-5 that provides opportunities for parents to learn more about technology skills their children are learning (e.g., career options in STEM areas, overview of ICAC, and summary of student projects). At this event, a yearly citywide competition among students also will be held that is an expanded version of the weeklong showcase event during the student summer camps.
- Surveying of students twice a year in intervention schools.
- Surveying of teachers at Summer Institutes and then at the end of the academic year.
- Coding and entry of survey data; coding of interview and observational data.
- Data analysis to examine the specific aims (SA) noted above:
- The impact of ICAC on teacher computing confidence and utilization (SA 3.b).
- Assess the effects of (1) teacher XO training on student computing confidence and utilization (SA 3.b), (2) training on changes in interest in STEM careers (SA 3.c), and (3) XO training on student engagement (SA 3.a).
- A quasi-experimental comparison of intervention and non-intervention schools to assess intervention effects on student achievement (SA 3.a).
- Survey of parents attending the yearly ICAC showcase to assess effects on parental attitudes toward STEM careers and computing (SA 3.d).
The proposed research has the potential for broad impact by leveraging technology in BCS to influence over 8,000 students in the Birmingham area. By targeting 4th and 5th grade students, we expect to impact STEM engagement and preparedness of students before they move into a critical educational and career decision-making process. Further, by bolstering student computer and STEM knowledge, ICAC will impart highly marketable skills that prepare them for the 81% of new jobs that are projected to be in computing and engineering in coming years (as predicted by the US Bureau of Labor Statistics).3 Through its formative and summative assessment, ICAC will offer intellectual merit by providing teachers throughout the US with insights into how computers can be used to integrate the elementary STEM curriculum. ICAC will develop a model for using computers to enhance STEM education across the curriculum while instilling a culture among BCS schools where computing is viewed as a tool for learning.
The Role of Educative Curriculum Materials in Supporting Science Teaching Practices with English Language Learners
This project aims to determine whether curricula designed to support teacher and student learning have positive impacts on teacher knowledge, attitudes, and instructional practices; to what degree educative curricula help teachers with more and less experience teaching ELLs and how level of teaching experience relates to teacher knowledge, attitudes, and instructional practices; and the effects of the educative curricula in high implementation settings on ELLs knowledge and attitudes in science, and developing English proficiency.
Design and Use of Illustrations in Test Items as a Form of Accommodation for English Language Learners in Science and Mathematics Assessment
This project investigates how vignette illustrations minimize the impact of limited English proficiency on student performance in science tests. Different analyses will determine whether and how ELL and non-ELL students differ significantly on the ways they use vignettes to make sense of items; whether the use of vignettes reduces test-score differences due to language factors between ELL and non-ELL students; and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.
This exploratory project within the Contextual Strand (Challenge a) addresses validity in the assessment of science and mathematics for English language learners (ELLs), and the urgent need for effective testing accommodations for ELLs. Motivation for this investigation originated from a previous, NSF-funded project on the testing of ELLs. We observed that items which were accompanied by illustrations tended to be responded correctly by a higher percentage of students than items without illustrations. We will investigate the factors that are relevant to designing and using a new form of accommodation in the assessment of science and mathematics for ELLs—vignette illustrations.
This three-year project will be guided by four research questions: What principles underlie the effective design of science and mathematics test items with illustrations in ways that minimize limited English proficiency as a factor that prevents ELLs from understanding the items? Is the presence of an illustration a moderator in students’ understanding test items? If so, Is the effect due to the simple presence of a graphical component or due to characteristics of the illustrations that are created based on principled design? Does the presence of an illustration have a different effect on the performance of ELLs and the performance of non-ELL students?
We expect to be able to: 1) identify the role of illustrations in the cognitive activities elicited by vignette-illustrated items; 2) determine whether any differences between performance on vignette-illustrated items and other kinds of items are due to the this form of accommodation’s capacity to address language as a construct-irrelevant factor; 3) identify the set of practical and methodological issues that are critical to properly developing and using vignette-illustrated items; and 4) propose a set of documents and procedures for the systematic and cost-effective design and development of vignette-illustrated items.
We will test ELL and non-ELL students with items of three types (vignette-illustrated items whose illustrations are designed systematically, vignette-illustrated items whose illustrations are created arbitrarily, and items without illustrations) at two levels of distance to the enacted curriculum (close and distal). Diverse forms of analysis will allow us to determine whether and how ELL and non-ELL students differ on the ways in which they use vignettes to make sense of items, whether the use of vignettes reduces test score differences due to language factors between ELL and non-ELL students, and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.
Intellectual merit. This project will provide information that will help to advance our understanding in two assessment arenas: effective accommodations for ELLs, and item development practices. While illustrations are frequently used in test items, there is not guidance in the assessment development literature on how to approach illustrations. Furthermore, the value of illustrations as a resource for ensuring that ELL students understand what a given item is about and what the item asks them to do has not been systematically investigated. Semiotics, cognitive psychology, and linguistics and socio-cultural theory are brought together to develop systematic procedures for developing illustrations as visual supports in tests. Understanding the role that images play in test taking is relevant to devising more effective ways of testing students. While this project aims to improve testing accommodations practices for ELLs, knowledge gained from it will inform test development practices relevant to all student populations.
Broader impact. We expect outcomes of this project to contribute to enhanced practice in both classroom and large-scale assessment. The push for including ELLs in large-scale testing programs with accountability purposes is not corresponded by effective testing accommodation practices. Many testing accommodations used by national and state assessment programs are not defensible, are not effective, or are improperly implemented. Vignette illustrations have the potential to become a low-cost, easy-to-implement form of testing accommodation for ELLs. Results form this investigation will allow us to identify a set of principles for the proper design and use of vignette illustrations as a form of testing accommodation for ELLs. The project is important not only because it explores the potential of an innovative form of accommodation but because it uses a systematic procedure for designing that form of accommodation.
The goal of this project is to accelerate the progress of early-career and pre-service science teachers from novice to expert-like pedagogical reasoning and practice by developing and studying a system of discourse tools. The tools are aimed at developing teachers' capabilities in shaping instruction around the most fundamental science ideas; scaffolding student thinking; and adapting instruction to diverse student populations by collecting and analyzing student data on their thinking levels.
Closing the Math Achievement Gap for English Language Learners: Technology Resources for Pre-algebra
The project addresses the relatively poor mathematics achievement of students who are not proficient in English. It includes research on how English language learners in beginning algebra classes solve math word problems with different text characteristics. The results of this research inform the development of technology-based resources to support ELLs’ ability to learn mathematics through instruction in English, including tutorials in math vocabulary, integrated glossaries, and interactive assistance with forming equations from word problem text.
This project is based on the assumption that teachers often make modifications to curriculum; reordering, skipping or adding lessons, changing an \"exploration\" into a lecture, and so on. This project pursues three related questions: What types of modifications do teachers make (and why), which types of modifications best help students learn, and how do teachers' modifications change in response to professional development activities designed to help them become more attuned to students' thinking?
An Architecture of Intensification: Building a Comprehensive Program for Struggling Students in Double-period Algebra Classes
This project is carrying out a research and development initiative to increase the success rates of our most at-risk high school students—ninth-grade students enrolled in algebra classes but significantly underprepared for high school mathematics. It will also result in new understandings about effective approaches for teaching mathematics to struggling students and about effective ways for implementing these approaches at scale, particularly in urban school districts.
Intensified Algebra I, a comprehensive program used in an extended-time algebra class, helps students who are one to two years behind in mathematics become successful in algebra. It is a research and development initiative of the Charles A. Dana Center at The University of Texas at Austin, the Learning Sciences Research Institute at the University of Illinois at Chicago, and Agile Mind, that transforms the teaching of algebra to students who struggle in mathematics. Central to the program is the idea that struggling students need a powerful combination of a challenging curriculum, cohesive, targeted supports, and additional well-structured classroom time. Intensified Algebra I seeks to addresses the need for a robust Algebra I curriculum with embedded, efficient review and repair of foundational mathematical skills and concepts. It aims to address multiple dimensions of learning mathematics, including social, affective, linguistic, and cognitive. Intensified Algebra I uses an asset-based approach that builds on students’ strengths and helps students to develop academic skills and identities by engaging them in the learning experience. The program is designed to help struggling students succeed in catching up to their peers, equipping them to be successful in Algebra I and their future mathematics and science courses.
Learning Science as Inquiry with the Urban Advantage: Formal-Informal Collaborations to Increase Science Literacy and Student Learning
This project hypothesizes that learners must have access to the real work of scientists if they are to learn both about the nature of science and to do inquiry themselves. It explores the question "How can informal science education institutions best design resources to support teachers, school administrators, and families in the teaching and learning of students to conduct scientific investigations and better understand the nature of science?"
The American Museum of Natural History and Michigan State University propose a research and development project focused on DR-K12 challenge #2 and the hypothesis that learners must have access to the real work of scientists if they are to learn both about the nature of science and to do inquiry themselves. The overarching questions that drive this project are: How can informal science education institutions best design resources to support teachers, school administrators, and families in the teaching and learning of students to conduct scientific investigations and better understand the nature of science? How are these resources then used, and to what extent and in what ways do they contribute to participants’ learning? How are those resources then used for student learning? Answering these questions will involve the use of existing and new resources, enhancement of existing relationships, and a commitment to systematically collect evidence. Urban Advantage (UA) is a middle school science initiative involving informal science education institutions that provides professional development for teachers and hands-on learning for students to learn how to conduct scientific investigations. This project will (1) refine the UA model by including opportunities to engage in field studies and the use of authentic data sets to investigate the zebra mussel invasion of the Hudson River ecosystem; (2) extend the resources available to help parents, administrators, and teachers understand the nature of scientific work; and (3) integrate a research agenda into UA. Teaching cases will serve as resources to help teachers, students, administrators, and families understand scientific inquiry through research on freshwater ecosystems, and—with that increased understanding—support student learning. Surveys, observations, and assessments will be used to document and understand the effects of professional development on teachers, students, administrators, and parents. The study will analyze longitudinal, multivariate data in order to identify associations between professional development opportunities for teachers, administrators, and parents, their use of resources to support their own learning and that of students, middle school teachers’ instructional practices, and measures of student learning.
This group assembled at the 2009 DR K-12 PI Meeting to begin planning support activities that will meet the needs of PIs over the duration of the network. As a first step, CADRE mapped the grants that have been funded in the first three cohorts of DR K-12. This map gives us a landscape to plan supports and resources that add value to the work of PIs as well as to identify expertise and needs that exist within the current DR K-12 community. Currently, efforts focused on specific topics within the field of assessment are getting under way. We will soon establish separate group workspaces for each of these groups so that members cans communicate, post resources, and collaobrate on products that are specific to each initiative.
New Measurement Paradigms: Psychometric Methods for Technology-based Assessments:
The use of technology in measurement of learning presents new opportunities for better understanding of what students know and can do. It vastly expands the range of variables that may be measured, and allows for measurement in real time. But these advantages also bring psychometric challenges because the majority of methods we have used in educational measurement to date have grown out of paper and pencil technologies and are not well suited to the complex, multidimensional, on-the-fly scoring and reporting that are required in cutting edge technology-based assessments.
This group will focus on how technologies can be applied to enhance what we know about students’ knowledge and abilities and explore what methods of assessment development, scoring and reporting are needed to make those assessments as effective as possible. It will examine what reliability and validity mean in the new contexts created by advanced use of technology, and what methods might be applied in determining those qualities. For more information, please email CADRE@edc.org.
Assessment and Pedagogical Practices in Diagnostic Learning Environments
Several DR-K12 projects are exploring issues related to creating and fostering classroom environments that promote learning through assessment and pedagogical practices that support the formative use of assessments. Drawing on approaches applied in our own projects, we will examine synergies and distinctions among design principles and frameworks used to guide the development of assessments (e.g., evidence-centered design, facet-based/knowledge-in-pieces perspectives, learning progressions, and universal design for learning). We also will articulate the goals and purposes of different types of assessments and discuss how and when these assessments can be used most effectively in science classrooms, particularly in classrooms with culturally and linguistically diverse learners. A critical component of this group’s work will be to articulate strategies for validating assessments designed to promote learning in science classrooms. Products of this group will include one or more presentations at DR-K12 PI meetings and papers describing ideas evolving from this collaboration among leaders of NSF-funded projects. For more information, please email CADRE@edc.org.