Effects of Algorithmic Transparency in Bayesian Knowledge Tracing on Trust and Perceived Accuracy
Kimberly Williamson, Rene Kizilcec
Jul 01, 2021 16:00 UTC+2
—
Session E1
—
Zoom link
Keywords: Bayesian Knowledge Tracing, Data Visualization, Explainable AI
Abstract:
Knowledge tracing algorithms such as Bayesian Knowledge Tracing (BKT) can provide students and teachers with helpful information about their progress towards learning objectives. Despite the popularity of BKT in the research community, the algorithm is not widely adopted in educational practice. This may be due to skepticism from users and uncertainty over how to explain BKT to them to foster trust. We conducted a pre-registered 2x2 survey experiment (n=170) to investigate attitudes towards BKT and how they are affected by verbal and visual explanations of the algorithm. We find that ostensible learners prefer BKT over a simpler algorithm, rating BKT as more trustworthy, accurate, and sophisticated. Providing verbal and visual explanations of BKT improved confidence in the learning application, trust in BKT and its perceived accuracy. Findings suggest that people's acceptance of BKT may be higher than anticipated, especially when explanations are provided.