Estimating the Intelligent Tutor Effects on Specific Posttest Problems
Adam Sales, Ethan Prihar, Neil Heffernan, John Pane
Jun 30, 2021 16:00 UTC+2
—
Session B1
—
Zoom link
Keywords: Causal impact estimates, multilevel modeling, intelligent tutoring systems
Abstract:
This paper drills deeper into the documented effects of the Cognitive Tutor Algebra I and ASSISTments intelligent tutoring systems by estimating their effects on specific problems. We start by describing a multilevel Rasch-type model that facilitates testing for differences in the effects between problems and precise problem-specific effect estimation without the need for multiple comparisons corrections. We find that the effects of both intelligent tutors vary between problems--the effects are positive for some, negative for others, and undeterminable for the rest. Next we explore hypotheses explaining why effects might be larger for some problems than for others. In the case of ASSISTments, there is no evidence that problems that are more closely related to students' work in the tutor displayed larger treatment effects.