ABSTRACT

Understanding how learning unfolds moment-to-moment remains a central challenge in capturing real-time learning processes. Much of the field has focused on aggregated behavioral signals or prediction of outcomes, often overlooking the fine-grained temporal dynamics through which learning processes emerge. This work introduces a multimodal, temporally explicit framework for modeling reading comprehension by integrating eye tracking, EEG, and textual features at the level of individual fixations during naturalistic reading.

We analyzed data from 116 participants reading five long expository texts while eye movements and EEG were recorded concurrently. We then constructed fixation-level, multivariate time series that included gaze (fixation duration), neural responses (fixation-related potentials), and lexical properties (surprisal and word frequency). We applied multivariate vector autoregression (mlVAR) to model how these signals unfold over time, both within fixations (contemporaneous) and across fixations. We also examined how comprehension, reading time, prior knowledge, and mind wandering covary with network connectivity.

Across models, robust lexical-gaze relationships were observed, with surprisal positively associated with fixation duration and word frequency negatively associated with fixation duration, both within and across fixations. These patterns extend classic findings from reading research into a temporal, multivariate setting. Neural associations were more sensitive to modeling choices, suggesting that brain-behavior coupling may depend on how temporal structure is represented. At the participant level, comprehension and reading time were associated with differences in contemporaneous network connectivity, indicating that variation in learning outcomes corresponds to differences in multimodal coordination.

Overall, this work positions temporal network modeling as a promising direction. By integrating multimodal signals at a fine temporal scale in naturalistic settings, it provides a framework for studying learning as a dynamic, evolving process rather than a static outcome. Beyond reading comprehension. This approach offers a generalizable methodology for modeling temporal dependencies across modalities in a wide range of educational contexts.

Keywords

© 2026 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.