The Use of Illustrations in Large-Scale Science Assessment: A Comparative Study

In this paper, we report on a study that compares state, national, and international assessment programs as to the characteristics and functions of the illustrations used in their science test items. We used our conceptual framework for examining the characteristics of illustrations in science items (Solano-Flores & Wang, 2009, 2011) to code the illustrations of samples of items. We examined the statistical significance of differences in the frequencies of different illustration variables observed in samples of science items from assessments from two countries (China and the U.S,) in four science areas, physics, chemistry, biology, and earth and space science. We observed statistically significant differences between the numbers of features in the illustrations originated in China and the illustrations originated in the U.S. Illustrations from China tended to have more varied and complex characteristics than their U.S. counterparts. We discuss the implications of these findings in the design of science items in assessment projects that involve culturally and linguistically diverse populations in both the U.S. and in the context of international test comparisons.