Detecting Careless Responding to Assessment Items in a Virtual Learning Environment Using Person-fit Indices and Random Forest
Sanaz Nazari, Walter Leite, Anne Huggins-Manley
Jun 30, 2021 20:40 UTC+2
—
Session PS1
—
Gather Town
Keywords: Careless responding, Person-fit index, Random forest classifier, Virtual learning environment
Abstract:
Careless responding and keeping students motivated for different tests have been common problems in many areas, especially in education. This study’s objective was to demonstrate a novel approach to detect careless responding using person-fit indices developed within the field of psychometrics combined with a random forest. The data used was obtained from various tests in the Math Nation virtual learning platform. The result of person-fit indices as previously used measures of careless responding as well as the result of a random forest classifier to capture careless responding were compared by Receiver Operating characteristic (ROC) analysis and the area under the curve (AUC). The result showed that random forest combined with person-fit indices outperformed person-fit indices directly in detecting careless responding. Some important applications of this method for applied researchers are discussed in the conclusion section.