ABSTRACT

We examined dynamic relationships between eye gaze, brain activity, and text characteristics during reading comprehension, a ubiquitous educational activity. We applied multilevel vector autoregression (mlVAR) to naturalistic reading data to model across-fixation (temporal) and within-fixation (contemporaneous) relationships among eye movements, EEG, and lexical features of the word being attended. We examined different approaches to encoding the multimodal time series data and how prior knowledge, reading time, mind wandering, and comprehension outcomes were related to network dynamics. Our data were from 116 participants who engaged in a complex learning task involving five long, connected texts on scientific research methods. Across model configurations, lexical features consistently predicted fixation duration, highlighting text-driven control of reading behavior, whereas EEG-gaze interactions emerged under certain model configurations. Measures of cognitive states, mainly comprehension and reading time, predicted within-fixation network connectivity. These findings demonstrate that temporally sensitive multimodal models provide a useful framework for characterizing moment-to-moment reading dynamics.

Keywords

1. INTRODUCTION

Reading comprehension remains a key skill in education and everyday life: whether learning from a textbook in class or reading an article at home for leisure. Decades of research have shown that comprehension emerges from dynamic interactions between text characteristics, individual differences in the reader, and the reading context [72, 42, 43]. Eye-tracking has been a key modality used for understanding encoding and processing of words, sentences, and text [27, 27, 56, 12], and even for predicting comprehension outcomes and attentional states during reading [12, 24, 5, 30]. This is due to its tight link to moment-by-moment cognitive processing, a phenomenon deemed the Eye-Mind Link. For example, fixations (when gaze is maintained in a single spot) and saccades (quick movements between fixations) are sensitive to textual features such as word frequency and complexity [59, 55, 61]. Consequently, a weakening of this gaze-text link can be used to predict internal mental states associated with disrupted comprehension such as mind wandering [5, 30, 62]. The eye-mind link has been applied in real-time interventions to prompt readers to re-read text if they mind wander [49], or adapt texts when comprehension difficulty is detected [67]. Yet, relying on a single modality (gaze) limits these models’ ability to explain and predict the complex, temporally unfolding cognitive processes that occur during reading comprehension.

Neural measures offer a more direct window into the brain (arguably the locus of the “mind") and have been used to investigate the cognitive processes that support reading comprehension. Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide rich information about encoding, semantic processing, and integration during reading and language processing [20, 6, 31]. Importantly, EEG measures the brain’s electrical response at the millisecond scale enabling capture of fast moving brain dynamics. Here, EEG studies have focused on capturing neural dynamics on a single word using Rapid Serial Visual Presentation (RSVP) paradigm, where words are sequentially presented one at a time. Given the lack of ecological validity of this approach, some studies have used fixation-related potentials (FRPs), where EEG activity is time-locked to fixations during more naturalistic reading [19].

However, FRP studies also generally rely on highly-controlled text stimuli, including artificially constructed sentences, often restricting analysis to a subset of experimentally manipulated target words that should elicit predicted lexical effects [20, 66, 34]. This includes removing certain short function words from analysis even though they make up a decent proportion of viewed words [10, 34], but have limited lexical information (very high frequency and low surprisal). Within these contexts, characteristic components of the FRP have been associated with linguistic variables such as surprisal (lack of predictability) and word frequency [79, 66]. However, such designs prioritize experimental control at the expense of ecological validity. They obscure how neural activity dynamically couples with eye movements to support the broader cognitive aspects of reading, such as how comprehension is constructed while reading a long connected text, and how it is affected by lapses in attention. FRP studies also tend to quantify fixations in isolation, largely ignoring the temporal dynamics across multiple fixations.

The present study focuses on modeling the temporal dynamic coupling between eye movements, neural activity (EEG), and text characteristics during natural reading of long, connected texts. We extract multivariate timeseries consisting of fixation-related EEG responses in parallel with fixation durations and linguistic features of fixated words. To capture how these features jointly evolve over time, we use multi-level vector autoregression (mlVAR) to model temporal (across fixation) and contemporaneous (within-fixation) dependencies of these timeseries across readers. This is one of the first attempts (to our knowledge) to simultaneously investigate temporal dependencies between eye, brain, and the text during relatively naturalistic reading settings.

2. BACKGROUND AND RELATED WORK

2.1 Reading Comprehension

Reading comprehension involves interacting levels of processing that unfold over time: perceptual intake and encoding of a word [36], accessing a word’s literal meaning [52], constructing a representation of the text and updating it with prior knowledge from memory [40], eventually leading to a rich mental representation of what was read [42, 43]. Reading is thus a highly interactive process where readers must continually retain relevant information, reactivate earlier representations, and suppress information that becomes temporally irrelevant [80], all while continuing perceptual uptake and encoding of words as they proceed through the text.

A widely-used framework for reading comprehension distinguishes among three levels of mental representation: the surface code, textbase, and situation model [76]. The surface code reflects a literal encoding of the words and syntax of the text. The textbase captures the meaning of sentences independent of their specific wording. Finally, the situation model integrates meaning across the entire text, combining what was read with the reader’s prior knowledge [35]. Cognitive theories such as Kintsch’s Construction-Integration model [41, 43] explain how higher-level inferences emerge from these initial, more literal meanings of the text. This means disruptions in constructing the surface code or textbase can have downstream effects on developing the situation model [68]. For instance, if a reader mind wanders while reading a sentence containing key information, this hinders their ability to later form inferences pertaining to that information [69]. Taken together, this highlights reading comprehension as a temporally evolving process in which moment-to-moment encoding accumulates to support the construction of a coherent mental model.

2.2 Eye Movements during Reading

Linguistic processing must be tightly coordinated with the visual system to coordinate perceptual uptake [39]. As such, eye movements index and track the cognitive processes underlying reading comprehension as they unfold. Decades of research on gaze control during reading support a strong coupling between eye movements and textual features [55, 58, 27, 61]. For example, fixations occur when information is encoded, and are shorter on more frequent words [59, 58] and longer on less predictable or surprising words [55]. This relationship can even be temporally complex, as illustrated by the "spillover" effect when fixations are longer following difficult words [55, 61]. Saccades are the quick movements between fixations, and regressions are saccades to previous sections of the text and occur approximately 10%-25% of the time [57]. More difficult texts elicit more and longer fixations, as well as more regressions [14] reflecting a breakdown in encoding or integration processes [63].

This coupling between eye movements and online mental processing is deemed the Eye-Mind Link [54, 60]. Beyond indexing lower-level lexical processing, eye movements have been leveraged to predict higher-level comprehension and integration processes, such as detecting knowledge incongruence in text via characteristic regressive patterns [77]. Skim reading, typically associated with decreased comprehension [57], is characterized by fewer and shorter fixations and fewer regressions [50]. In contrast, mind wandering ("zoning out"), an attentional state also robustly linked with impaired comprehension, has a less consistent gaze signature. Mind wandering been associated with fewer fixations and regressions (similar to skim reading) and longer fixations (similiar to reading difficult texts) in some studies [4, 30, 62], but an increase in fixation count in others [75]. Other studies have explored mind wandering in regards to coupling, finding a decreased sensitivity to linguistic and lexical changes in the text[2, 11].

Given these relationships, machine learning using eye movements have been able to predict shallow [24, 73, 15, 46] and deep comprehension of texts [12], as well as mind wandering [5, 30, 62] during learning contexts. However, to address the inherent ambiguity of information in individual fixations, these models aggregate gaze features across predefined windows or entire pages of text, losing fine-grained dynamic temporal structure, which is the present focus.

2.3 EEG Correlates of Linguistic Processing

Investigating neural activity offers the potential to go beyond coarser grained eye-gaze analyses towards differentiating processing within and across fixations, enabling more precise characterization of reading dynamics. Controlled experiments using EEG have identified neural correlates of language comprehension. Event-related potentials (ERPs) measured on the scalp reflect neurons activating in synchrony [45] resulting in consistent voltage fluctuations comprising positive and negative peaks unfolding over several hundred milliseconds [45]. Linguistic ERPs are traditionally measured in tightly controlled experiments where words are presented in isolation for a brief fixed interval using RSVP [23, 20]. Whereas this approach allows for the precise delineation of the neural response to single words [21], it does not reflect naturalistic reading settings.

A fixation-related potential (FRP), on the other hand, is an ERP that is time-locked to the onset of a fixation during natural reading and allows for a richer measure of perceptual and cognitive processing that occurs during processing of a word in context [19, 20]. FRPs require more in-depth processing to remove overlapping neural responses to successive words [16, 71] and have been used during natural reading to investigate the relationship between EEG and eye movement measures [33, 23].

Early ERP components during reading are primarily associated with visual processing and encoding, and are sensitive to low-level lexical features. For instance the P1 and N1 components which occur 80-120 ms post-fixation show smaller responses for more frequent words [66]. The P2 (150-300 ms post-fixation) has been linked to early orthographic and phonological processing as well as lexical access, with amplitudes modulated by word frequency [20]. Later components, such as the well-studied N400 (300-500 ms post-fixation), reflect higher-level semantic processing and integration [79] and are sensitive to semantic predictability and contextual fit [44]. These components can overlap across successive fixations, demonstrating that visual, lexical, and semantic processes are dynamically coordinated as reading and comprehension unfold [19, 20].

The ability of FRPs to capture neural processing within a single fixation is particularly valuable in distinguishing complex cognitive states even when fixation durations are similar. In particular, longer fixation durations can arise for different reasons– either because the reader is disengaged from the text [62], or because they are allocating resources towards processing a difficult word [55, 26]. Eye movements alone cannot distinguish these cases. In contrast, EEG provides a direct window into the neural processes unfolding during a fixation, allowing covert states to be differentiated at a finer temporal scale. For example, Broadway et al. [9] showed that mind wandering during reading is associated with attenuated early visual ERP components such as P1, consistent with perceptual decoupling from external input [64]. The strength of these early neural responses predicted subsequent reading comprehension, linking momentary engagement to later comprehension outcomes. Boudewyn et al. further demonstrated that during story listening, mind wandering reduced the sensitivity of the N400 to contextual linguistic variables such as word position and surprisal [7]. More recently, Southwell et al. [74], replicated this finding involving natural reading. Using multivariate linear modeling applied to concurrent EEG and eye movement recordings, they found that mind wandering led to a decreased modulation of the N400 by surprisal, such that the amplitude of the N400 was larger (more negative voltage) during mind wandering. This further suggests a greater sensitivity to word meaning when readers are attending to the text. Together, these findings highlight the value of incorporating EEG into temporal models of reading, as neural activity captures fixation-by-fixation fluctuations in attentional and linguistic processing that are not observable from eye movements alone.

3. NOVELTY AND CONTRIBUTIONS

Reading is a temporally evolving process in which eye movements, neural activity, and textual properties continuously interact. However, most prior multimodal reading research involving the eye, brain, and text has focused on individual within-fixation associations examined under controlled experimental conditions rather than the fixation time series as a whole.

Accordingly, the goal of the current study is to provide an exploratory analysis of temporal reading dynamics across modalities and to examine how these dynamics are related to cognitive processes underlying reading. To do so, we implemented multilevel vector autoregression [29, 28], a relatively new modeling approach that enables joint estimation of across-fixation (temporal) and within-fixation (contemporaneous) relationships among eye movements, EEG activity, and lexical properties. mlVAR explicitly separates lagged dependencies across successive fixations from instantaneous coupling within a fixation, allowing these two sources of structure to be estimated simultaneously.

At the fixation level, mlVAR is implemented as a system of linear mixed-effects models in which features from the previous fixation are used to predict features at the current fixation, allowing for group-level network patterns to be estimated while capturing participant-specific deviations. By modeling multiple modalities in a unified temporal framework, mlVAR provides an exploratory lens on how gaze behavior, neural activity, and lexical properties co-vary over time. This approach enables our first research question (RQ1): What are the within- and across-fixation relationships between EEG, gaze behavior, and text during naturalistic reading?

The second aim of the study is to explore how cognitive processes modulate these dynamic relationships. Specifically, we ask (RQ2): How do mind wandering, comprehension, prior knowledge, and reading time relate to variation in reading dynamics? These factors are treated as sources of variability in the strength of dependencies among eye movements, neural activity, and text. While some of these cognitive variables have been explored at the level of single fixations, they have not been jointly modeled across modalities or across successive fixations.

Finally, because most FRP approaches do not model fixation sequences over time, they do not require preserving temporal continuity. Accordingly, it is common for these approaches to exclude stop words (short words that include articles, prepositions, conjunctions, and pronouns, e.g., "a", "the", "in", "or"), and restrict analyses to content words (e.g., nouns, words). However, stop words and their associated fixations are an integral part of the reading process and how readers move through, pause on, or transition across these words may carry meaningful information about cognitive state. For example, prolonged fixations on stop words may reflect mind wandering. We therefore introduce (RQ3): How do different strategies for representing stop words into the fixation time series shape the temporal effects and cognitive influences observed in the model.

To address this, we compare three modeling strategies (see Figure 4). Given that the definition of the \(t-1\) fixation is the key modeling detail in mlVAR, we tried three modeling strategies that differed in how stop words were incorporated into the definition of the \(t-1\) fixation. In Model 1 we used only transitions from content words to content words, making no assumptions about stop words and excluding transitions that were originally to or from stop words. This keeps the focus to content words, most clearly mirroring traditional FRP analysis. Model 2 assumes stop words are minimally processed and barely attended to, effectively skipping them by removing stop word fixations and modeling a sequence of fixations represented as between fixation content words only. In other other words, the \(t-1\) fixation is defined as the previous content word attended to. Lastly, Model 3 retained stop words in the sequence by imputing them with the lowest surprisal and highest frequency values, allowing the fixation sequence to include both content words and stop words. These modeling strategies will unveil how important assumptions shape which aspects of coupling are emphasized. See Figure 2 for a visualization of each modeling strategy.

The novelty of this work lies in applying mlVAR to multimodal fixation-level, real-world reading to explore patterns that can inform future theory-driven and application-focused modeling efforts. To our knowledge, no study has investigated the temporal dynamics of eye gaze, EEG, and text characteristics during naturalistic reading of long expository educational texts. As a result, our findings can be useful in educational contexts.

4. DATASET

This study was part of a larger study and only aspects germane to the present research are discussed here.

4.1 Participants

A total of 116 participants (mean age = 22 +/- 5.7 (SD), 70% female, 1% identifying as another gender) were students from a large public university in the western United States. Participants were compensated $20 per hour via Amazon gift cards. All procedures were approved by the university’s Institutional Review Board, and written informed consent was obtained from all participants prior to participation.

4.2 Data Acquisition

4.2.1 Eye Tracking

High-resolution, binocular gaze was tracked using the desktop-mounted SR Research EyeLink 1000+ with a sampling rate of 1000 Hz. Stimuli were displayed on a 23.8", 1920x1080 pixel display and viewed at a distance of 90 cm. Participants stabilized their head using a chin rest.

4.2.2 EEG

EEG was recorded at 1000 Hz using a LiveAmp wireless amplifier with 8 electrodes on an ActiCAP (Brain Products GmbH, Munich, Germany cite). A custom layout based on the 10-5 system was designed to optimize scalp coverage while also working around other sensors present on the cap. The montage included two standard mid-line locations (CPz and FCz), 3 pairs of symmetrically-located channels at frontal (AFF5h and AFF6h), centro-parietal (CCP5h and CCP6h), and parieto-occipital (PPO9h and PPO10h) locations. Reference was at Cz and Ground adjacent to FPz.

4.3 Reading Task & Measures

4.3.1 Texts and Reading Procedure

Participants read five expository texts on the computer monitor, totaling in 5306 words over 50 pages (mean 106 words per page). Each text consisted of ten pages (screens) of text and participants could advance through the pages at their own pace, but could not return to pages already read. Texts were presented in a random order, and each focused on a topic of behavioral research methods: Hypothesis, Bias, Causal Claims, Validity, and Variables. The texts had a mean Flesch-Kincaid grade level of 13.2, indicating a slightly advance reading level suitable for college students [32]. Participants spent an average time of 27.6 (8.3 SD) minutes reading all texts. On average, 57.3% \(\pm \) 3.4% of words on a page were content words.

4.3.2 Reading Time

Page-level reading times were calculated from the beginning of the button press that initiates the page, to the next button press that leaves the page. Mean reading time per person was calculated by average page-level reading times across the entire text. Pages with short reading times (<10 seconds) were excluded, as they likely reflect insufficient engagement with the text rather than meaningful reading. Participants spent an average of 36 seconds (16 SD) per page.

4.3.3 Mind Wandering Probes

We used a probe-caught method previously validated to track mind wandering [65, 78]. With this method, participants indicate if they were mind wandering on the previous page via thought-probes presented throughout the study. Before beginning to read, participants are given the definition of mind wandering and instructions on how to respond to the probe. Mind wandering, or "zoning out", was defined as “not thinking about what you are actually reading, but thinking about something else altogether”. Participants are asked to be as honest as possible, and informed that reporting mind wandering does not affect their progress in the study. Probes occurred twice per text on pseudo random pages, for a total of ten probes throughout the reading (see 4.3.1). When participants advanced to the next page, they would first be prompted with a yes or no question asking if they were mind wandering on the previous page. The average mind wandering rate was 47% (24% SD).

4.3.4 Comprehension Questions

Reading comprehension was measured via three different types of assessments that target different levels of comprehension. Rote questions were four-item multiple choice targeting factual knowledge explicitly presented in the text. Inference questions consisted of a statement reflecting either a true or false inference that can be made by extracting implicit information from a specific single sentence on the page. Finally, integration questions were four-item multiple-choice questions assessing deeper, integrative knowledge across multiple pages.

These comprehension questions were administered at three different time points, but not all questions were administered at each time point. Questions were administered 1) during texts, meaning the questions appeared after reading the corresponding page; 2) after an individual text was read; 3) after all five texts were administered. Rote and inference questions were administered at all time points, while integration questions were only administered during texts and after all texts were completed (time points 1 and 2). Questions always appeared in the order of Rote, Inference, Integration to reflect the increasing levels of processing, and were preceded by the mind wandering probes (see Section 4.3.3). Additionally, pages that triggered comprehension questions were selected pseudorandomly with basic constraints, such that no questions never appeared on the first page or adjacent pages of the text. The specific questions administered to each participant were selected pseudorandomly from a main question pool without repetition. Mean comprehension across all questions was 68% (10% SD).

4.3.5 Prior Knowledge Assessment

Participants completed the prior knowledge assessment prior to beginning the reading task. The participants had to match six definitions relating to scientific methods with their concept names. All definitions and names were randomly ordered (Confound, Placebo, Dependent Variable, External Validity, Control Group, and A priori). Correct answers were summed. Mean prior knowledge score were 68% (17% SD).

5. DATA PROCESSING AND MODELING

5.1 Time series measures

5.1.1 Lexical Variables

Two lexical features were extracted per-word: surprisal and the log of word frequency. Surprisal for each word was computed as the negative log probability assigned to that word after inputting its page into a pre-trained GPT-2 small language model, as this model has been shown to align closest with human reading times [51]. English word frequency was calculated using the nltk package then log-transformed. Lexical variables were only defined for content words in this dataset; surprisal and word frequency values for stop words were not computed. In Model 1 lexical features for stop words were kept as missing. In Model 2, they were removed. Finally, in Model 3, lexical features were imputed to be the minimum surprisal and maximum frequency. Surprisal for content words in the text ranged between 0 and 26.5, with a mean and SD of 5.3 \(\pm \) 3.5. Log word frequency ranged from -17.98 to -2.92, with a mean of -10 and SD of 2.1. Figure 3 shows distributions of lexical variables.

5.1.2 Gaze Preprocessing

Fixations, blinks, and saccades were identified from the raw time series with EyeLink’s event-detection algorithm (velocity threshold of 30 degs/sec and acceleration threshold of 9500 \(\mathrm {deg}/\mathrm {s}^2\)). The right eye was used in the analysis if available, otherwise the left eye was used. The first and final fixations on each page were removed, as they are likely to be confounded with visual evoked potentials and motor artifacts (1.5% of fixations). Fixations with extreme durations (>600 or <50 ms) were also removed (5.1%), and pages with fewer than ten fixations were excluded (0.07%). The analysis focuses on relationship between lexical features, EEG, and gaze duration, so fixations not on words (no lexical information) or on punctuation were removed (5.12% of fixations). A total of 252795 fixations were included in the analyses, 28.37% of which were on stop words.

5.1.3 EEG Processing and FRP Extraction

EEG was time-locked to stimuli via trigger pulses generated at page transitions and other task-related events. Then EEG and eye-tracker data was aligned using a custom timestamp interpolation algorithm in Matlab R2021a [38] that accounted for timing drift between recording systems and accommodated missing triggers. For EEG preprocessing, line noise at 60 Hz was removed using the Zapline algorithm [17], and slow drifts were removed via robust detrending [18]. Bad channels were identified with the PREP pipeline [3] and interpolated from neighboring electrodes. All data was subsequently referenced to the average over all electrodes.

FRPs were estimated using a linear deconvolution approach implemented as a regression model to correct for potentially overlapping neural effects time-locked to word fixations [16, 22, 70]. For each participant and EEG channel, a linear model was fit to the continuous EEG time series. The model included categorical regressors marking the onsets of fixation events (separately for fixations on and off words), and additional regressors for blink and button-press events. This minimal model was chosen to estimate fixation-locked FRPs without imposing assumptions about cognitive or textual effects on the EEG.

Mean component amplitudes were extracted within predefined time windows and scalp regions of interest. This includes three canonical ERP components: P1 (70-120 ms) , measured over posterior visual regions; P2 (140-280 ms) measured over central and fronto-central regions; and N400 (300-500 ms), measured over central regions. These components and regions are selected based on prior literature and ensure separability given the limited spatial resolution of the EEG montage. For each fixation, component features were computed as the mean voltage within the corresponding time window and scalp region.

5.2 Multilevel Vector Autoregression

Multilevel vector autoregression (mlVAR) is a network analysis method that combines vector autoregression with linear mixed-effects modeling [8, 29]. Vector autoregression is the process of predicting a vector of variables at time \(t\) (index in sequence) using the same vector at time \(t-n\), where \(n\) is lag. Combined with linear mixed effects models, this allows modeling of temporal dependencies among multiple variables while accounting for the nested structure of the data (fixations within pages, and pages within individuals). We used the R package "mlVAR" (version 0.5.2) for all analysis [28].

5.2.1 Temporal Networks

First, mlVAR estimates temporal networks by sequentially predicting the vector of word-level EEG, gaze, and lexical variables for participant \(i\) at time \(t\) (denoted \(Y_{i,t}\)) from the same set of vectors at the previous time point \(t-1\) using a mixed-effects formulation:

\(B\) is the fixed effects matrix that contains the average estimates of population-level lagged effects between all variables. \(U_{i}\) is the random-effects matrix that captures participant-specific deviations from these temporal relationship. \(\epsilon _{i,t}\) is a matrix of residual error term. Each row of \(B\) corresponds to a separate regression model predicting one variable at time \(t\) from all variables at time \(t-1\).

The resulting fixed-effects coefficients then form the directed temporal networks. In these networks, a significant edge between two variables indicates that the "earlier" variable (\(t-1\)) significantly predicts the "later" one (\(t\)), after controlling for all other variables. Statistical significance was determine using a threshold of \( p < 0.05\), consistent with prior work [25, 37].

5.2.2 Contemporaneous Networks

In addition to temporal effects, mlVAR estimates contemporaneous networks. These capture associations between variables occurring at the same time point after accounting for lagged temporal dependencies. Contemporaneous networks are computed from the residual covariance structure of the temporal models. This reflects relationships between variables that co-occur within the same fixation but are not explained by prior states of the system (temporal effects or other variables).

Let \(\varepsilon _{it}\) denote the vector of residuals obtained from the temporal model for participant \(i\) at time \(t\). Contemporaneous associations are modeled by predicting each residual from the remaining residuals at the same time point using a mixed effects formulation:

where \(\varepsilon ^*_{it}\) denotes the vector of residual excluding the outcome variable currently being predicted. The fixed-effects matrix \(H\) contains population-level contemporaneous associations between variables, while \(G_i\) captures participant-specific deviations from these associations. The term \(e_{it}\) represents residual error.

Each row of \(H\) corresponds to a regression predicting one variable’s residual at fixation \(t\) from the residuals of all other variables at the same time point. The resulting fixed-effects coefficients from an undirected contemporaneous network, in which an edge between two variables reflects their partial association within the same fixation, after controlling for all lagged temporal effects and the variance shared with all other modeled variables. Conceptually, whereas temporal edges represent predictive, directional associations across successive fixations, contemporaneous edges capture instantaneous co-fluctuations between gaze, EEG, and lexical variables that arise within a single fixation and are not attributable to prior dynamics.

5.2.3 Selection of Focal Network Edges

While all edges between all variables were modeled in mlVAR, we focus on a subset of cognitively-relevant edges in both networks which address our research questions. Specifically, in the temporal network, we examine directed edges from text \(\rightarrow \) EEG and text \(\rightarrow \) fixation duration (gaze), which capture how lexical properties (surprisal and word frequency) are related to fixation duration and FRPs on the subsequent word. We additionally consider gaze \(\rightarrow \) EEG, reflecting how the amount of visual attention allocated to a word modulates subsequent neural processing, as well as EEG \(\rightarrow \) gaze edges, which capture how neural indices of processing are associated with attentional allocation on the following word.

In the contemporaneous network, we included the same edges but without directional components: text \(\leftrightarrow \) EEG, text \(\leftrightarrow \) gaze, and gaze \(\leftrightarrow \) EEG. Figure 4F shows the contemporaneous networks with these focal edges visualized. These fall into only three categories because contemporaneous edges are undirected and reflect symmetric associations between modalities that co-occur within a fixation.

Edges directed into lexical feature were excluded, as these features are properties of the text rather than of the reader and therefore do not reflect cognitive dynamics. Self-loops (edges which a variable predicts itself at the next time step) were also excluded, as they primarily reflect temporal autocorrelation rather than theoretically informative cross-modal associations. EEG self-loops in particular are difficult to interpret, as correlations across successive FRPs may arise from shared baseline activity, slow drifts, or differences in overall signal amplitude rather than meaningful cognitive processing across fixations.

By restricting analysis to focal cross-modal edges, we emphasize relationships that reflect dynamic interactions between text properties, eye movements, and neural responses during reading. Figure 4D illustrates the fourteen direct focal edges considered in the temporal network.

6. RESULTS

6.1 Temporal and Contemporaneous Effects

We first address RQ1 by examining network structure across models. Table 2 shows the significant edges and their weights. Overall, core lexical-to-gaze effects emerged within-fixations, and across fixations. These effects were consistent in direction across modeling approaches. EEG-related effects, in contrast, appeared only under certain modeling conditions.

Across both temporal and contemporaneous networks, surprisal to fixation duration had a positive effect and word frequency to fixation duration had a negative effect. These relationships are robust and were observed across all three models. The direction of these effects align with previous research examining within-fixation patterns. Specifically, longer fixations are typically observed for words requiring deeper processing: either because they are low frequency words or are surprising, unpredictable words given the sentence context [53, 55]. The fact that these relationships were observed in the temporal models suggests a spillover effect in which word properties for fixation t influence gaze patterns for fixation t+1.

Two additional edges involving EEG features emerged in particular networks and model configurations. In particular, the temporal network in had a negative edge between mean P2 and subsequent fixation duration: as mean voltage within the P2 window (140-280 ms after fixation onset) became more positive, fixation duration of the next word decreases. Increase in the P2 magnitude has been previously linked to an increase in early word-level processing within-fixations [1, 20], but has been less explored temporally. Interestingly, this association is only present in Model 3, which contains stop words that usually have shorter fixation durations. Taken together, this effect may reflect either reduced attention to the upcoming word following larger P2 response or a greater likelihood that the subsequent fixation lands on a word (like a stop word) that typically requires less processing.

Surprisal also showed a positive within-fixation association with the mean N400 in the contemporaneous networks for Models 2 and 3. That is, as surprisal increased, the mean value in the N400 window (300-500 ms) became more positive (less pronounced negative amplitude). This conflicts with previous research showing higher surprisal linked to an larger negative peak in the N400 [44]. However we use a mean voltage value within the analysis window. It is possible the mean value captures too much surrounding activity in the FRP, pulling the value more positive when shifts in the timing of the N400 peak occur. Additionally, imputing stop words affects the surprisal distribution in ways not traditionally modeled. Because stop words have very low surprisal, assigning them imputed values changes which surprisal values are most common in the data, so the surprisal-N400 relationship is estimated from a different mix of words than typical ERP studies.

Although individual edge magnitudes are small, this is expected in a multivariate time series of fixation-level data where effects reflect incremental, conditional associations after accounting for strong autoregressive and cross-modal dependencies. Small estimates at the fixation level can accumulate across the thousands of fixations per participant. Importantly, these effects were highly stable as evidenced by the following permutation tests.

6.1.1 Permutation Testing

To assess the robustness of the estimated network structures, we conducted a participant-level permutation subsampling analysis. Twenty five random subsamples of fifty participants each were drawn without replacement, and the mlVAR model was fit for each. Temporal and contemporaneous networks were extracted from each fitted model, and edge weights were aggregated across permutations to form empirical distributions for each network connection. We conducted this analysis using Model 1 as an illustration. Edges identified as significant in Model 1 were verified with a 95% confidence interval of their permutation distribution excluding zero. This procedure evaluates whether observed network dependencies reflect stable population-level relationships rather than effects driven by a small subset of participants, providing a robustness of the mlVAR results.

Permutation testing on Model 1 consistently showed the same significant edges. In the temporal network, the 95% confidence interval for the significant edges were [0.010, 0.014] for Surprisal \(\rightarrow \) Fixation Duration and [-0.022, -0.018] for Word Frequency \(\rightarrow \) Fixation Duration. In the contemporaneous network, the corresponding intervals were [0.018, 0.020] Surprisal \(\leftrightarrow \) Fixation Duration for and [-0.017, -0.015] for Word Freq \(\leftrightarrow \) Fixation Duration. These confidence intervals that do not overlap with 0 (null effect) indicate that the observed associations are reliable and robust despite their small magnitudes.

6.2 Cognitive Variables as Predictors of Network Connectivity

To address RQ2 we examined cognitive variables as predictors of connectivity in the temporal and contemporaneous networks for each model. Specifically, we leveraged the multilevel structure of the mlVAR model to extract participant-specific edge estimates. Because temporal edges are estimated using hierarchical linear mixed effects models, we can extract participant-level deviations from the group-level network. We extracted the fourteen participant-level temporal edges for each individual, for each relevant directed connection in the network as well as the 11 focal contemporaneous edges. We then investigated how participant-level differences in comprehension outcomes, mind wandering rates, prior knowledge, and reading time predicted total connectivity in the focal edges in each network (Figure 4). We calculated total connectivity as the sum of the absolute value of each focal edge (14 edges in temporal network and 11 edges in the contemporaneous network). We used the following linear regression model:

Results are shown in Table 3. There were no significant associations between the cognitive variables and total connectivity in the temporal networks. The contemporaneous networks, however, had positive effects of Comprehension in Models 1 and 2, and a negative effect of reading time in Model 3.

These results suggest that higher comprehension is associated with stronger within-fixation coupling between modalities when stop words are excluded (Models 1 and 2). One interpretation is that readers who achieve better comprehension exhibit more tightly coordinated processing across gaze, neural, and lexical signals. Greater total connectivity may reflect a more cognitively coupled state in which the modalities co-vary more strongly [48]. Importantly, this association does not imply a directional relationship between comprehension and connectivity, but instead highlights that stronger comprehension tends to co-occur with denser contemporaneous structure in the multimodal network. That absence of this pattern when stop words are retained further suggests that this coupling is more evident during fixations on content-bearing words, which entails integration of lower-level of processing into a coherent mental model of the text including making bridging inferences from previous parts of the text and also elaborative inferencing from long-term memory [47].

In contrast, when stop words are included (Model 3), longer reading times are associated with weaker contemporaneous connectivity. Reading time is a complex measure that can indicate skimming, scanning, or deep reading strategies [13], each of which is reflected in distinct gaze patterns. Increased reading time can also function as a compensatory strategy when comprehension difficulty occurs [13]. Readers may compensate by slowing down or through regressive (i.e., re-reading) eye movements [54], which would increase reading times. Here, longer reading times were associated with weaker contemporaneous network connectivity, i.e. weaker coupling between modalities occurring within-fixation. One interpretation is an association between comprehension difficulties (longer reading time to repair difficulties) and weaker network structure.

Lastly, neither prior knowledge nor mind wandering were predictors of network connectivity. It is possible these variables show selective differences across network edges rather than changes in overall connectivity.

7. DISCUSSION

The primary aim of this work was to examine how multimodal temporal modeling can help uncover moment-to-moment eye-mind coordination between gaze, fixation-related potentials, and text characteristics during reading. Our findings illustrate how fixation-level mlVAR can serve as an exploratory tool to examine temporal and within-fixation coupling (RQ1), and examine cognitive associations on network connectivity (RQ2), and compare different approaches for representing the time series (RQ3). Although, we focused on the specific domain of reading comprehension here, our we aim for the overall modeling approach to be informative and potentially useful to related domains in educational data mining.

7.1 Main Findings

Regarding RQ1, the observed temporal and contemporaneous networks align with classic accounts of the eye-mind link that emphasize a close coupling between linguistic processing and eye-movement behavior during reading. Moment-to-moment gaze dynamics were sensitive to the lexical properties of the text, consistent with long-standing views that fixation duration reflects ongoing word-level processing. Additionally, the temporal effects across fixations extend prior static within-fixation analyses by showing that these established lexical-gaze relationships might be anticipatory or signal spillover effects. Neural associations were less stable and more sensitive to how fixation sequences were represented.

We explored RQ2 by investigating effects of participant-level cognitive variables (mind wandering, reading time, prior knowledge, comprehension) on total network connectivity. Comprehension and reading time showed effects in the contemporaneous network only: reading comprehension scores predicted connectivity whereas reading time negatively predicted it. Overall, these effects are consistent with theoretical perspectives highlighting a tight coupling between the eye, brain, and text required for successful comprehension, and that increased reading times might signal comprehension difficulties and repair strategies. Rather than treating the models as competing methods, we interpret them as complementary lenses on the same underlying reading process, each making different assumptions. Differences in cognitive effects across models reflect how these assumptions shape which aspects of multimodal coupling are emphasized, rather than indicating that one model is more correct than others. More research is needed to examine how the reported effects are sensitive to modeling choices.

Finally, to address RQ3, we compared results across three modeling strategies that differed in how function words (stop words) were treated in the determination of \(t-1\) fixations for temporal modeling. Model 1 preserved only the content-to-content word transitions that were originally present in the fixation sequence, making no assumptions about stop words and excluding any transitions to or from stop words. Model 2 treated stop words as minimally processed and short in duration, effectivity skipping them by removing stop-word fixations so that all modeled transitions occurred between content words. Model 3 retained stop words in the sequence by imputing them with the lowest surprisal and highest frequency values, allowing fixation sequences to include both content and function words.

Core lexical-gaze relations were stable to different stop word treatments, while neural-gaze and lexical-neural associations were more model dependent. Given the additional P2 to fixation duration and surprisal to N400 edges in Model 3, neural coupling might be more sensitive to how stop words are handled, potentially due to prediction mechanisms that are not captured in the eye gaze sequence. Additionally, a negative effect of reading time on total contemporaneous connectivity was found only in Model 3. Because stop words are associated with minimal semantic processing and brief fixations, including them may reduce the coherence of within-fixation multimodal structure, though this needs to be examined further.

7.2 Limitations

A primary limitation of our study is the interpretation of findings given the nature of the analyses. Although we examined associations between cognitive measures and network properties, the results do not support claims about underlying mechanisms or directional relationships (e.g. whether higher comprehension stems from stronger connectivity or stronger connectivity stems from better comprehension). Addressing such questions requires analytic approaches beyond the correlational framework used here. Second, the temporal resolution of the model is constrained to the fixation-level. Cognitive processes involved in reading do not always unfold at the timescale of individual fixations, and lag-1 dependencies might fail to capture longer-range dynamics across words, sentences, or pages. Additionally, cognitive measures were aggregated at the participant-level, which may obscure moment-to-moment variability in cognitive states. Lastly, our data were collected in a specific lab setting where participants wore a neural cap, read expository texts, wore masks (due to Covid-19). Thus, it is unclear whether our findings can be generalized to more ecological reading scenarios.

7.3 Concluding Remarks

The eye-mind link is inherently dynamic, relying on moment-to-moment coordination between text, gaze, and neural activity. The results presented here suggest that temporal modeling of fixation sequences can recover well-established patterns from the reading literature while extending them to richer, multimodal, relationships and elucidating temporal effects. This approached helps assess whether relationships typically observed in isolation or within-fixations are observed in the context of full-length sequences with temporal interdependencies across fixations. While lexical properties show robust associations with subsequent fixation duration, incorporating EEG alongside gaze measures offers additional information for tracking cognitive states during reading. Our findings also suggest that low-level details in how the underlying time series are encoded can have major implications on the conclusions. Lastly, while we focused on reading comprehension and a specific multimodal configuration of signals here, the overall mlVAR approach can be applied to any set of modalities collected across a range of domains, thereby providing new insights into the learning process.

8. ACKNOWLEDGMENTS

This research was supported by the National Science Foundation (DRL 1920510). The opinions expressed are those of the authors and do not represent views of the funding agencies.

9. REFERENCES

1. T. Baccino and Y. Manunta. Eye-fixation-related potentials: Insight into parafoveal processing. Journal of Psychophysiology, 19(3):204–215, 2005.
2. R. Biedert, J. Hees, A. Dengel, and G. Buscher. A robust realtime reading-skimming classifier. In Proceedings of the Symposium on Eye Tracking Research and Applications, ETRA ’12, page 123–130, New York, NY, USA, 2012. Association for Computing Machinery.
3. N. Bigdely-Shamlo, T. Mullen, C. Kothe, K.-M. Su, and K. A. Robbins. The prep pipeline: standardized preprocessing for large-scale eeg analysis. Frontiers in neuroinformatics, 9:16, 2015.
4. R. Bixler and S. D’Mello. Automatic gaze-based detection of mind wandering with metacognitive awareness. In International Conference on User Modeling, Adaptation, and Personalization, pages 31–43. Springer, 2015.
5. R. Bixler and S. D’Mello. Automatic gaze-based user-independent detection of mind wandering during computerized reading. User Modeling and User-Adapted Interaction, 26(1):33–68, 2016.
6. I. Blank, Z. Balewski, K. Mahowald, and E. Fedorenko. Syntactic processing is distributed across the language system. Neuroimage, 127:307–323, 2016.
7. M. A. Boudewyn. Half-listening or zoned out? it’s about the same: the impact of attentional state on word processing in context. Cognitive neuroscience, 14(3):107–114, 2023.
8. L. F. Bringmann, N. Vissers, M. Wichers, N. Geschwind, P. Kuppens, F. Peeters, D. Borsboom, and F. Tuerlinckx. A network approach to psychopathology: New insights into clinical longitudinal data. PloS one, 8(4):e60188, 2013.
9. J. M. Broadway, M. S. Franklin, and J. W. Schooler. Early event-related brain potentials and hemispheric asymmetries reveal mind-wandering while reading and predict comprehension. Biological Psychology, 107:31–43, 2015.
10. M. P. Broderick, A. J. Anderson, G. M. Di Liberto, M. J. Crosse, and E. C. Lalor. Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narrative speech. Current Biology, 28(5):803–809, 2018.
11. G. Buscher, A. Dengel, and L. Van Elst. Query expansion using gaze-based feedback on the subdocument level. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pages 387–394, 2008.
12. M. Caruso, C. E. Peacock, R. Southwell, G. Zhou, and S. K. D’Mello. Going deep and far: Gaze-based models predict multiple depths of comprehension during and one week following reading. International Educational Data Mining Society, 2022.
13. R. P. Carver. Reading rate: Theory, research, and practical implications. Journal of Reading, 36(2):84–95, 1992.
14. A. E. Cook and W. Wei. What can eye movements tell us about higher level comprehension? Vision, 3(3):45, 2019.
15. L. Copeland, T. Gedeon, and S. Caldwell. Effects of text difficulty and readers on predicting reading comprehension from eye movements. In 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pages 407–412, 2015.
16. T. Cornelissen, J. Sassenhagen, and M. L.-H. Vo. Improving free-viewing fixation-related eeg potentials with continuous-time regression. Journal of Neuroscience Methods, 313:77–94, 2019.
17. A. De Cheveigné. Zapline: A simple and effective method to remove power line artifacts. NeuroImage, 207:116356, 2020.
18. A. De Cheveigné and D. Arzounian. Robust detrending, rereferencing, outlier detection, and inpainting for multichannel data. NeuroImage, 172:903–912, 2018.
19. F. Degno and S. Liversedge. Eye movements and fixation-related potentials in reading: A review. vision, 4 (1), 11. Vision, 4(1), 2020.
20. F. Degno, O. Loberg, and S. P. Liversedge. Co-registration of eye movements and fixation—related potentials in natural reading: Practical issues of experimental design and data analysis. Collabra: Psychology, 7(1):18032, 2021.
21. K. A. DeLong, T. P. Urbach, and M. Kutas. Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nature neuroscience, 8(8):1117–1121, 2005.
22. O. Dimigen and B. V. Ehinger. Regression-based analysis of combined eeg and eye-tracking data: Theory and applications. Journal of vision, 21(1):3–3, 2021.
23. O. Dimigen, W. Sommer, A. Hohlfeld, A. M. Jacobs, and R. Kliegl. Coregistration of eye movements and eeg in natural reading: analyses and review. Journal of experimental psychology: General, 140(4):552, 2011.
24. S. K. D’Mello, R. Southwell, and J. Gregg. Machine-learned computational models can enhance the study of text and discourse: A case study using eye tracking to model reading comprehension. Discourse processes, 57(5-6):420–440, 2020.
25. O. V. Ebrahimi, J. Burger, A. Hoffart, and S. U. Johnson. Within-and across-day patterns of interplay between depressive symptoms and related psychopathological processes: a dynamic network approach during the covid-19 pandemic. BMC medicine, 19(1):317, 2021.
26. S. F. Ehrlich and K. Rayner. Contextual effects on word perception and eye movements during reading. Journal of verbal learning and verbal behavior, 20(6):641–655, 1981.
27. R. Engbert, A. Nuthmann, E. M. Richter, and R. Kliegl. Swift: a dynamical model of saccade generation during reading. Psychological review, 112(4):777, 2005.
28. S. Epskamp, M. K. Deserno, L. F. Bringmann, and M. Veenman. Package ‘mlvar’, 2022.
29. S. Epskamp, L. J. Waldorp, R. Mõttus, and D. Borsboom. The gaussian graphical model in cross-sectional and time-series data. Multivariate behavioral research, 53(4):453–480, 2018.
30. M. Faber, R. Bixler, and S. K. D’Mello. An automated behavioral measure of mind wandering during computerized reading. Behavior Research Methods, 50(1):134–150, 2018.
31. E. Fedorenko, J. Duncan, and N. Kanwisher. Language-selective and domain-general regions lie side by side within broca’s area. Current Biology, 22(21):2059–2062, 2012.
32. R. Flesch. A new readability yardstick. Journal of applied psychology, 32(3):221, 1948.
33. S. L. Frank and A. Aumeistere. An eye-tracking-with-eeg coregistration corpus of narrative sentences. Language Resources and Evaluation, 58(2):641–657, 2024.
34. S. L. Frank and R. M. Willems. Word predictability and semantic similarity show distinct patterns of brain activity during language comprehension. Language, Cognition and Neuroscience, 32(9):1192–1203, 2017.
35. A. C. Graesser, M. Singer, and T. Trabasso. Constructing inferences during narrative text comprehension. Psychological review, 101(3):371, 1994.
36. J. Grainger and P. J. Holcomb. Watching the word go by: On the time-course of component processes in visual word recognition. Language and linguistics compass, 3(1):128–156, 2009.
37. L. M. Hamel, R. Moulder, F. W. Harper, L. A. Penner, T. L. Albrecht, and S. Eggly. Examining the dynamic nature of nonverbal communication between black patients with cancer and their oncologists. Cancer, 127(7):1080–1090, 2021.
38. T. M. Inc. Matlab version: R2021a, 2021.
39. M. A. Just and P. A. Carpenter. Eye fixations and cognitive processes. Cognitive psychology, 8(4):441–480, 1976.
40. P. Kendeou and P. Van Den Broek. The effects of prior knowledge and text structure on comprehension processes during reading of scientific texts. Memory & cognition, 35(7):1567–1577, 2007.
41. W. Kintsch. The role of knowledge in discourse comprehension: a construction-integration model. Psychological review, 95(2):163, 1988.
42. W. Kintsch. Text comprehension, memory, and learning. American psychologist, 49(4):294, 1994.
43. W. Kintsch. Comprehension: A paradigm for cognition. Cambridge university press, 1998.
44. M. Kutas and K. D. Federmeier. Thirty years and counting: finding meaning in the n400 component of the event-related brain potential (erp). Annual review of psychology, 62(1):621–647, 2011.
45. S. J. Luck. An introduction to the event-related potential technique. MIT press, 2014.
46. P. Martínez-Gómez and A. Aizawa. Recognition of understanding level and language skill using measurements of reading behavior. In Proceedings of the 19th international conference on Intelligent User Interfaces, pages 95–104, Haifa Israel, Feb. 2014. ACM.
47. D. S. McNamara. If integration is the keystone of comprehension: Inferencing is the key. Discourse Processes, 58(1):86–91, 2021.
48. C. Mills, A. Graesser, E. F. Risko, and S. K. D’Mello. Cognitive coupling during reading. Journal of experimental psychology: General, 146(6):872, 2017.
49. C. Mills, J. Gregg, R. Bixler, and S. K. D’Mello. Eye-mind reader: An intelligent reading interface that promotes long-term comprehension by detecting and responding to mind wandering. Human–Computer Interaction, 36(4):306–332, 2021.
50. H. Miyata, Y. Minagawa-Kawai, S. Watanabe, T. Sasaki, and K. Ueda. Reading speed, comprehension and eye movements while reading japanese novels: Evidence from untrained readers and cases of speed-reading trainees. PloS one, 7(5):e36091, 2012.
51. B.-D. Oh and W. Schuler. Why does surprisal from larger transformer-based language models provide a poorer fit to human reading times? Transactions of the Association for Computational Linguistics, 11:336–350, 2023.
52. C. J. Price, C. J. Moore, G. W. Humphreys, and R. J. Wise. Segregating semantic from phonological processes during reading. Journal of cognitive neuroscience, 9(6):727–733, 1997.
53. K. Rayner, J. Ashby, A. Pollatsek, and E. D. Reichle. The effects of frequency and predictability on eye fixations in reading: implications for the ez reader model. Journal of Experimental Psychology: Human Perception and Performance, 30(4):720, 2004.
54. K. Rayner, K. H. Chace, T. J. Slattery, and J. Ashby. Eye movements as reflections of comprehension processes in reading. Scientific studies of reading, 10(3):241–255, 2006.
55. K. Rayner and S. A. Duffy. Lexical complexity and fixation times in reading: Effects of word frequency, verb complexity, and lexical ambiguity. Memory & cognition, 14(3):191–201, 1986.
56. K. Rayner and G. W. McConkie. What guides a reader’s eye movements? Vision research, 16(8):829–837, 1976.
57. K. Rayner, A. Pollatsek, J. Ashby, and C. Clifton Jr. Psychology of reading. Psychology Press, 2012.
58. K. Rayner, E. D. Reichle, and A. Pollatsek. Eye movementcontrol in reading: An Overview and Model. Eye guidance in reading and scene perception, pages 243–268, 1998.
59. K. Rayner, T. J. Slattery, D. Drieghe, and S. P. Liversedge. Eye movements and word skipping during reading: effects of word length and predictability. Journal of Experimental Psychology: Human Perception and Performance, 37(2):514, 2011.
60. E. D. Reichle. Computational models of eye-movement control during reading: Theories of the" eye-mind" link. Cognitive Systems Research, 2006.
61. E. D. Reichle, K. Rayner, and A. Pollatsek. The e-z reader model of eye-movement control in reading: Comparisons to other models. Behavioral and Brain Sciences, 26(4):445–476, 2003.
62. E. D. Reichle, A. E. Reineberg, and J. W. Schooler. Eye movements during mindless reading. Psychological science, 21(9):1300–1310, 2010.
63. E. D. Reichle, T. Warren, and K. McConnell. Using ez reader to model the effects of higher level language processing on eye movements during reading. Psychonomic bulletin & review, 16(1):1–21, 2009.
64. J. W. Schooler, J. Smallwood, K. Christoff, T. C. Handy, E. D. Reichle, and M. A. Sayette. Meta-awareness, perceptual decoupling and the wandering mind. Trends in cognitive sciences, 15(7):319–326, 2011.
65. A.-L. Schubert, G. T. Frischkorn, and J. Rummel. The validity of the online thought-probing procedure of mind wandering is not threatened by variations of probe rate and probe framing. Psychological research, 84(7):1846–1856, 2020.
66. S. C. Sereno, C. J. Hand, A. Shahid, I. G. Mackenzie, and H. Leuthold. Early eeg correlates of word frequency and contextual predictability in reading. Language, cognition and neuroscience, 35(5):625–640, 2020.
67. V. J. Shute and D. Zapata-Rivera. Adaptive educational systems. In P. J. Durlach and A. M. Lesgold, editors, Adaptive Technologies for Training and Education, pages 7–27. Cambridge University Press, Cambridge, 2012.
68. J. Smallwood, D. J. Fishman, and J. W. Schooler. Counting the cost of an absent mind: Mind wandering as an underrecognized influence on educational performance. Psychonomic bulletin & review, 14(2):230–236, 2007.
69. J. Smallwood, M. McSpadden, and J. W. Schooler. When attention matters: The curious incident of the wandering mind. Memory & cognition, 36(6):1144–1150, 2008.
70. N. J. Smith and M. Kutas. Regression-based estimation of erp waveforms: I. the rerp framework. Psychophysiology, 52(2):157–168, 2015.
71. N. J. Smith and M. Kutas. Regression-based estimation of erp waveforms: Ii. nonlinear effects, overlap correction, and practical considerations. Psychophysiology, 52(2):169–181, 2015.
72. C. Snow. Reading for understanding: Toward an R&D program in reading comprehension. Rand Corporation, 2002.
73. R. Southwell, J. Gregg, R. Bixler, and S. K. D’Mello. What eye movements reveal about later comprehension of long connected texts. Cognitive Science, 44(10):e12905, 2020.
74. R. Southwell, A. Kim, and S. D’Mello. (in review) eeg deconvolution modelling of naturalistic reading reveals surprisal effects modulated during mind wandering. In Review.
75. L. Steindorf and J. Rummel. Do your eyes give you away? a validation study of eye-movement measures used as indicators for mindless reading. Behavior research methods, 52(1):162–176, 2020.
76. T. A. Van Dijk, W. Kintsch, et al. Strategies of discourse comprehension. 1983.
77. M. L. van Moort, D. D. Jolles, A. Koornneef, and P. van den Broek. What you read versus what you know: Neural correlates of accessing context information and background knowledge in constructing a mental representation during reading. Journal of Experimental Psychology: General, 149(11):2084, 2020.
78. T. L. Varao-Sousa and A. Kingstone. Are mind wandering rates an artifact of the probe-caught method? using self-caught mind wandering in the classroom to test, and reject, this possibility. Behavior Research Methods, 51(1):235–242, 2019.
79. R. Wong, E. D. Reichle, and A. Veldre. Prediction in reading: A review of predictability effects, their theoretical implications, and beyond. Psychonomic Bulletin & Review, 32(3):973–1006, 2025.
80. M. Yeari. The role of working memory in inference generation during reading comprehension: Retention,(re) activation, or suppression of verbal information? Learning and Individual Differences, 56:1–12, 2017.

© 2026 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.