Detecting Careless Responding to Assessment Items in a Virtual Learning Environment Using Person-fit Indices and Random Forest
Abstract: Careless responding and keeping students motivated for different tests have been common problems in many areas, especially in education. This study’s objective was to demonstrate a novel approach to detect careless responding using person-fit indices developed within the field of psychometrics combined with a random forest. The data used was obtained from various tests in the Math Nation virtual learning platform. The result of person-fit indices as previously used measures of careless responding as well as the result of a random forest classifier to capture careless responding were compared by Receiver Operating characteristic (ROC) analysis and the area under the curve (AUC). The result showed that random forest combined with person-fit indices outperformed person-fit indices directly in detecting careless responding. Some important applications of this method for applied researchers are discussed in the conclusion section.