The project has had three major areas of focus: (1) Offering professional development to help elementary and 6th grade teachers become more responsive teachers, attending and responding to their students' ideas and reasoning; (2) Developing web-based resources (both curriculum and case studies) to promote responsive teaching in science; and (3) research how both teachers and students progress in their ability to engage in science inquiry.
This project implemented a facets-of-thinking perspective to design tools and practices to improve high school chemistry teachers' formative assessment practices. Goals are to identify and develop clusters of facets related to key chemistry concepts; develop assessment items; enhance the assessment system for administering items, reporting results, and providing teacher resource materials; develop teacher professional development and resource materials; and examine whether student learning in chemistry improves in classes that incorporate a facet-based assessment system.
Supported by research on students' preconceptions, particularly in chemistry, and the need to build on the knowledge and skills that students bring to the classroom, this project implements a facets-of-thinking perspective for the improvement of formative assessment, learning, and instruction in high school chemistry. Its goals are: to identify and develop clusters of facets (students' ideas and understandings) related to key high school chemistry concepts; to develop assessment items that diagnose facets within each cluster; to enhance the existing web-based Diagnoser assessment system for administering items, reporting results, and providing teacher resource materials for interpreting and using the assessment data; to develop teacher professional development and resource materials to support their use of facet-based approaches in chemistry; and to examine whether student learning in chemistry improves in classes that incorporate a facet-based assessment system.
The proposed work builds on two previously NSF-funded projects focused on designing Diagnoser (ESI-0435727) in the area of physics and on assessment development to support the transition to complex science learning (REC-0129406). The work plan is organized in three strands: (1) Assessment Development, consisting of the development and validation of facet clusters related to the Atomic Structure of Matter and Changes in Matter and the development and validation of question sets related to each facet cluster, including their administration to chemistry classes; (2) Professional Development, through which materials will be produced for a teacher workshop focused on the assessment-for-learning cycle; and (3) Technology Development, to upgrade the Diagnoser authoring system and to include chemistry facets and assessments.
Anticipated products include: (1) 8-10 validated facet clusters related to the Atomic Structure of Matter and Changes in Matter; (2) 12-20 items per facet cluster that provide diagnostic information about student understanding in relation to the facet clusters; (3) additional instructional materials related to each facet cluster, including 1-3 questions to elicit inital student ideas, a developmental lesson to encourage students' exploration of new concepts, and 3-5 prescriptive lessons to address persistent problematic ideas; and (4) a publically-available web-based Diagnoser for chemistry (www.Diagnoser.com), including student assessments and instructional materials.
This project builds and tests applications tied to the school curriculum that integrate the sciences with mathematics, computational thinking, reading and writing in elementary schools. The investigative core of the project is to determine how to best integrate computing across the curriculum in such a way as to support STEM learning and lead more urban children to STEM career paths.
Computer access has opened an exciting new dimension for STEM education; however, if computers in the classroom are to realize their full potential as a tool for advancing STEM education, methods must be developed to allow them to serve as a bridge across the STEM disciplines. The goal of this 60-month multi-method, multi-disciplinary ICAC project is to develop and test a program to increase the number of students in the STEM pipeline by providing teachers and students with curricular training and skills to enhance STEM education in elementary schools. ICAC will be implemented in an urban and predominantly African American school system, since these schools traditionally lag behind in filling the STEM pipeline. Specifically, ICAC will increase computer proficiency (e.g., general usage and programming), science, and mathematics skills of teachers and 4th and 5th grade students, and inform parents about the opportunities available in STEM-centered careers for their children.
The Specific Aims of ICAC are to:
SA1. Conduct a formative assessment with teachers to determine the optimal intervention to ensure productive school, principal, teacher, and student participation.
SA2. Implement a structured intervention aimed at (1) teachers, (2) students, and (3) families that will enhance the students’ understanding of STEM fundamentals by incorporating laptops into an inquiry-based educational process.
SA3. Assess the effects of ICAC on:
a. Student STEM engagement and performance.
b. Teacher and student computing specific confidence and utilization.
c. Student interest in technology and STEM careers.
d. Parents’ attitudes toward STEM careers and use of computers.
To enable us to complete the specific aims noted above, we have conducted a variety of project activities in Years 1-3. These include:
- Classroom observations at the two Year 1 pilot schools
- Project scaling to 6 schools in Year 2 and 10 schools in Year 3
- Semi-structured school administrator interviews in schools
- Professional development sessions for teachers
- Drafting of curriculum modules to be used in summer teacher institutes and for dissemination
- In-class demonstration of curriculum modules
- Scratch festivals each May
- Summer teacher institutes
- Student summer camps
- Surveying of teachers in summer institutes
- Surveying of teachers and students at the beginning and end of the school year
- Showcase event at end of student workshops
The specific ICAC activities for Years 2-5 include:
- Professional development sessions (twice monthly for teachers), to integrate the ‘best practices’ from the program.
- Working groups led by a grade-specific lead teacher. The lead teacher for each grade in each school will identify areas where assistance is needed and will gather the grade-specific cohort of teachers at their school once every two weeks for a meeting to discuss the progress made in addition to challenges to or successes in curricula development.
- ICAC staff and prior trained teachers will visit each class monthly during the year to assist the teachers and to evaluate specific challenges and opportunities for the use of XOs in that classroom.
- In class sessions at least once per month (most likely more often given feedback from Teacher Summer Institutes) to demonstrate lesson plans and assist teachers as they implement lesson plans.
- ICAC staff will also hold a joint meeting of administrators of all target schools each year to assess program progress and challenges.
- Teacher Summer Institutes – scaled-up to teachers from the new schools each summer to provide training in how to incorporate computing into their curriculum.
- Administrator sessions during the Teacher Summer Institutes; designed to provide insight into how the laptops can facilitate the education and comprehension of their students in all areas of the curriculum, discuss flexible models for physical classroom organization to facilitate student learning, and discussions related to how to optimize the use of computing to enhance STEM curricula in their schools. Student Summer Computing Camps – designed to teach students computing concepts, make computing fun, and enhance their interest in STEM careers.
- ICAC will sponsor a yearly showcase event in Years 2-5 that provides opportunities for parents to learn more about technology skills their children are learning (e.g., career options in STEM areas, overview of ICAC, and summary of student projects). At this event, a yearly citywide competition among students also will be held that is an expanded version of the weeklong showcase event during the student summer camps.
- Surveying of students twice a year in intervention schools.
- Surveying of teachers at Summer Institutes and then at the end of the academic year.
- Coding and entry of survey data; coding of interview and observational data.
- Data analysis to examine the specific aims (SA) noted above:
- The impact of ICAC on teacher computing confidence and utilization (SA 3.b).
- Assess the effects of (1) teacher XO training on student computing confidence and utilization (SA 3.b), (2) training on changes in interest in STEM careers (SA 3.c), and (3) XO training on student engagement (SA 3.a).
- A quasi-experimental comparison of intervention and non-intervention schools to assess intervention effects on student achievement (SA 3.a).
- Survey of parents attending the yearly ICAC showcase to assess effects on parental attitudes toward STEM careers and computing (SA 3.d).
The proposed research has the potential for broad impact by leveraging technology in BCS to influence over 8,000 students in the Birmingham area. By targeting 4th and 5th grade students, we expect to impact STEM engagement and preparedness of students before they move into a critical educational and career decision-making process. Further, by bolstering student computer and STEM knowledge, ICAC will impart highly marketable skills that prepare them for the 81% of new jobs that are projected to be in computing and engineering in coming years (as predicted by the US Bureau of Labor Statistics).3 Through its formative and summative assessment, ICAC will offer intellectual merit by providing teachers throughout the US with insights into how computers can be used to integrate the elementary STEM curriculum. ICAC will develop a model for using computers to enhance STEM education across the curriculum while instilling a culture among BCS schools where computing is viewed as a tool for learning.
(Previously listed under Award # 0918216)
Understanding Science provides an accurate portrayal of the nature of science and tools for teaching associated concepts. This project has at its heart a public re-engagement with science that begins with teacher preparation. To this end, its immediate goals are (1) improve teacher understanding of the nature of the scientific enterprise and (2) provide resources and strategies that encourage and enable K-16 teachers to incorporate and reinforce the nature of science throughout their science teaching.
This project aims to determine whether curricula designed to support teacher and student learning have positive impacts on teacher knowledge, attitudes, and instructional practices; to what degree educative curricula help teachers with more and less experience teaching ELLs and how level of teaching experience relates to teacher knowledge, attitudes, and instructional practices; and the effects of the educative curricula in high implementation settings on ELLs knowledge and attitudes in science, and developing English proficiency.
This project investigates how vignette illustrations minimize the impact of limited English proficiency on student performance in science tests. Different analyses will determine whether and how ELL and non-ELL students differ significantly on the ways they use vignettes to make sense of items; whether the use of vignettes reduces test-score differences due to language factors between ELL and non-ELL students; and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.
This exploratory project within the Contextual Strand (Challenge a) addresses validity in the assessment of science and mathematics for English language learners (ELLs), and the urgent need for effective testing accommodations for ELLs. Motivation for this investigation originated from a previous, NSF-funded project on the testing of ELLs. We observed that items which were accompanied by illustrations tended to be responded correctly by a higher percentage of students than items without illustrations. We will investigate the factors that are relevant to designing and using a new form of accommodation in the assessment of science and mathematics for ELLs—vignette illustrations.
This three-year project will be guided by four research questions: What principles underlie the effective design of science and mathematics test items with illustrations in ways that minimize limited English proficiency as a factor that prevents ELLs from understanding the items? Is the presence of an illustration a moderator in students’ understanding test items? If so, Is the effect due to the simple presence of a graphical component or due to characteristics of the illustrations that are created based on principled design? Does the presence of an illustration have a different effect on the performance of ELLs and the performance of non-ELL students?
We expect to be able to: 1) identify the role of illustrations in the cognitive activities elicited by vignette-illustrated items; 2) determine whether any differences between performance on vignette-illustrated items and other kinds of items are due to the this form of accommodation’s capacity to address language as a construct-irrelevant factor; 3) identify the set of practical and methodological issues that are critical to properly developing and using vignette-illustrated items; and 4) propose a set of documents and procedures for the systematic and cost-effective design and development of vignette-illustrated items.
We will test ELL and non-ELL students with items of three types (vignette-illustrated items whose illustrations are designed systematically, vignette-illustrated items whose illustrations are created arbitrarily, and items without illustrations) at two levels of distance to the enacted curriculum (close and distal). Diverse forms of analysis will allow us to determine whether and how ELL and non-ELL students differ on the ways in which they use vignettes to make sense of items, whether the use of vignettes reduces test score differences due to language factors between ELL and non-ELL students, and whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.
Intellectual merit. This project will provide information that will help to advance our understanding in two assessment arenas: effective accommodations for ELLs, and item development practices. While illustrations are frequently used in test items, there is not guidance in the assessment development literature on how to approach illustrations. Furthermore, the value of illustrations as a resource for ensuring that ELL students understand what a given item is about and what the item asks them to do has not been systematically investigated. Semiotics, cognitive psychology, and linguistics and socio-cultural theory are brought together to develop systematic procedures for developing illustrations as visual supports in tests. Understanding the role that images play in test taking is relevant to devising more effective ways of testing students. While this project aims to improve testing accommodations practices for ELLs, knowledge gained from it will inform test development practices relevant to all student populations.
Broader impact. We expect outcomes of this project to contribute to enhanced practice in both classroom and large-scale assessment. The push for including ELLs in large-scale testing programs with accountability purposes is not corresponded by effective testing accommodation practices. Many testing accommodations used by national and state assessment programs are not defensible, are not effective, or are improperly implemented. Vignette illustrations have the potential to become a low-cost, easy-to-implement form of testing accommodation for ELLs. Results form this investigation will allow us to identify a set of principles for the proper design and use of vignette illustrations as a form of testing accommodation for ELLs. The project is important not only because it explores the potential of an innovative form of accommodation but because it uses a systematic procedure for designing that form of accommodation.
This project aims to develop, pilot, and evaluate a model of instruction that advances the scientific literacy of high school students by involving them in science journalism, and to develop research tools for assessing scientific literacy and engagement. We view scientific literacy as public understanding of and engagement with science and technology, better enabling people to make informed science-related decisions in their personal lives, and participate in science-related democratic debates in public life.
For a more in-depth look at Scijourn, visit the project spotlight.
This project is designed to enhance understanding of how online professional development environments contribute to teach learning, changes in classroom practice and changes in student learning in comparison to face-to-face professional development. Using secondary school teachers learning to use a reformed-oriented environmental science curriculum, groups of teachers will be randomly assigned to one of three conditions: (1) traditional face-to-face workshop, (2)self-guided online professional development, or (3)online “short course” professional development guided by a facilitator.
This project is designing, developing, and testing a model that delivers effective teacher PD to in-service and preservice teachers to enable the successful implementation of engineering curricula. Research is performed to evaluate the impacts of the curricular materials and the teacher PD framework on classroom instructional practices and student learning, interests, and attitudes and to evaluate which curriculum components are most effective in promoting student learning and interest as a function of gender and ethnicity.