This project contributes to the small research base by exploring the validity of Technology-Enhanced Items (TEIs) in the context of elementary geometry. The project addresses three research questions: 1) To what extent are TEIs a valid measurement of geometry standards in the elementary grades?; 2) To what extent do TEIs provide an improved measurement compared to SR items?; and 3) What are the general characteristics of mathematics standards that might be better measured through TEIs?
Assessment developers, state departments of education, and national consortia have focused extensive efforts on including Technology-Enhanced Items (TEIs) on summative and formative assessments. TEIs have a number of potential benefits over traditional, selected-response (SR) items, including the potential to measure higher-level constructs, the reduction of the effects of test-taking skills and guessing, the capture of rich diagnostic information, the reduction of cognitive load from non-relevant constructs, and the engaging nature of their design. The first three benefits are true of constructed-response (CR) items, but TEIs have the added benefit of being automatically scored by computer. Despite the potential benefits of TEIs, and the strong push to include these types of items in assessments, there is a death of research on the validity of inferences made by TEIs and on whether TEIs provide improved measurement over traditional item types. The Validity of Technology-Enhanced Assessment in Geometry (VTAG) project contributes to the small research base by exploring the validity of TEIs in the context of elementary geometry.
The project addresses three research questions:
RQ1: To what extent are TEIs a valid measurement of geometry standards in the elementary grades?
RQ2: To what extent do TEIs provide an improved measurement compared to SR items?
RQ3: What are the general characteristics of mathematics standards that might be better measured through TEIs?
To address these research questions, the researchers develop 20 items (ten SR items and 10 TEIs) for each of the seven Common Core State Standards in fourth and fifth grade geometry. The researchers collect validity evidence using expert review, cognitive labs, and classroom administration of the items. The first two research questions are addressed by evaluating the validity of the items based on a variety of sources, including test content, internal structure, the relationship to other variables, and student response processes. To address the third research question, informed by the results of the prior two, the researchers use qualitative analysis to identify common themes of the standards that were identified as being better measured through TEIs.