Fine-Grained Versus Coarse-Grained Data for Estimating Time-on-Task in Learning Programming
Abstract: The time that students spend on assignments, i.e. time-on-task, has been used frequently in prior research to understand student affect, study habits, and course performance, among others. The choice for how time-on-task is calculated, however, is typically based on available data. This data can be very coarse-grained, such as the timestamps from students' assignment submissions. Using coarse-grained data to calculate time-on-task has limitations, such as not being able to determine whether students take breaks when working on an assignment. In this work, we analyze the differences between two time-on-task metrics, one based on coarse-grained data---in this case, student submissions---and one based on fine-grained data---in this case, students' keystrokes during an assignment. We compare these two metrics and examine how well they correlate to find out whether time-on-task based on coarse-grained data can be an accurate metric for understanding the time spent by students on an assignment. Our results show that the correlation between the two metrics that are supposed to measure the same underlying phenomena---time-on-task---is only weak to moderate. This suggests that fine-grained data might be needed to accurately estimate time-on-task.