Speeding up without Loss of Accuracy: Item Position Effects on Examinees’ Performance in University Exams
Leonardo Vida, Maria Bolsinova, Matthieu J. S. Brinkhuis
Jul 02, 2021 19:45 UTC+2
—
Session H3
—
Zoom link
Keywords: exam quality, computerized testing, item response time, item position effect, speededness, cognitive state, speed-accuracy trade-off
Abstract:
The quality of exams drives test-taking behavior of examinees and is a proxy for the quality of teaching. As most university exams have strict time limits, and speededness is an important measure of the cognitive state of examinees, this might be used to assess the connection between exams’ quality and examinees’ performance. The practice of randomization within university exams enables the analysis of item position effects within individual exams as a measure of speededness, and as such it enables the creation of a measure of the quality of an exam. In this research, we use generalized linear mixed models to evaluate item position effects on response accuracy and response time in a large dataset of randomized exams from an international research university. We find that there is an effect of item position on response time for most exams, but the same is not valid for response accuracy, which might be a starting point for identifying factors that influence speededness and can affect the mental state of examinees.