• Home
  • Schedule
  • Papers
  • Keynotes
  • Workshops
  • Master Page
  • Help

Execution Trace Based Feature Engineering To Enable Formative Feedback on Visual, Interactive Programs

Wengran Wang, Gordon Fraser, Tiffany Barnes, Chris Martens, Thomas Price

Jun 30, 2021 20:40 UTC+2 — Session PS1 — Gather Town

Keywords: novice programming, block-based programming, automated assessment, formative feedback, computer science education, code classification, execution trace mining, feature engineering

Abstract Paper
Abstract: Offering students immediate, formative feedback when they are programming can increase students' learning outcomes and self-efficacy. However, visual and interactive programs include dynamic user input and visual outputs that change over time, making it difficult to automatically assess students' code with traditional functional tests to offer this feedback. In this work, we introduce Execution Trace Based Feature Engineering (ETF), a feature engineering approach that extracts sequential patterns from execution traces, which capture the runtime behavior of students' code. We evaluated ETF on 162 students' code snapshots from a Pong game assignment in an introductory programming course, on a challenging task to predict students' success on fine-grained rubrics. We found that ETF achieves an average F1 score of 0.93 over 10 grading rubrics, which is 0.1-0.2 higher than a high-performing syntax-based code classification approach from prior work. These results show that ETF has strong potential to be used for code classification, to enable formative feedback for students' visual, interactive programs.

Video

Poster

Back to Top

© EDM 2021 Organization Committee