Automatically classifying student help requests: a multi-year analysis
Zhikai Gao, Collin Lynch, Sarah Heckman, Tiffany Barnes
Jun 30, 2021 15:00 UTC+2
—
Session A1
—
Zoom link
Keywords: Office Hour, Computer Science Education Research, Text Analysis, help-seeking request
Abstract:
As Computer Science has increased in popularity so too have class sizes and demands on faculty to provide support. It is therefore more important than ever for us to identify new ways to triage students questions, so that we can identify common problems, target students who need the most help, and better manage instructors' time. By analyzing office hours interaction data we can identify common patterns, and help to guide future help-seeking interactions. My Digital Hand (MDH) is an online ticketing system that allows students to post help requests, and for instructors to prioritize support and track common issues. In this research, we have collected and analyzed a corpus of student questions from across six semesters of a CS2 with a focus on object-oriented programming course. As part of this work, we grouped the interactions into five categories, analyzed the distribution of help requests, balanced the categories by Synthetic Minority Oversampling Technique(SMOTE), and trained an automatic classifier based upon LightGBM to automatically classify student requests. We found that over 69% of the questions were unclear or barely specified. We proved the stability of the model across semesters through Leave One Out and the target model achieves an accuracy of 91.8%. Finally, we find that online office hours can provide more help for more students.