ABSTRACT

Accurately tracing a learner’s knowledge state is a central challenge in many applications as it directly impacts educational downstream tasks such as recommendation, dropout prediction, and next-question correctness prediction. Prior research in learner modeling and knowledge tracing has shown that incorporating domain structure leads to more accurate estimates of learner knowledge, while parallel work has demonstrated that explicitly modeling epistemic uncertainty improves performance on these tasks. However, these two research directions have largely been explored independently. In particular, existing approaches rarely propagate uncertainty beyond the level at which learner evidence is directly observed, and domain-informed propagation strategies typically ignore uncertainty altogether. In this paper, we propose KT-DUSL, an uncertainty-aware learner model that jointly represents knowledge mastery, epistemic uncertainty, and domain structure within a unified framework. Building on subjective logic, we introduce a novel propagation policy that diffuses both mastery and uncertainty across the domain structure. This policy enables the model to better handle sparse interaction data. We evaluate KT-DUSL on large-scale datasets and show that propagating uncertainty through the domain structure yields improvements in next-question correctness prediction over uncertainty-agnostic and non-propagation-based baselines.

Keywords

1. INTRODUCTION

A learner model aims at tracing learners’ knowledge and can be understood as an explicit, machine-interpretable representation of an individual learner within an online learning environment [6]. The content of a learner model consists of a structured combination of learner data and inferred characteristics. Learner data includes relatively static attributes—such as demographic and educational background—as well as dynamic data derived from interactions with learning systems. Among the various learner characteristics that can be inferred from such data, the learner’s knowledge state is the most commonly modelled, typically referred to as a knowledge model [30]. Learner models can be used as an internal component of an Intelligent Tutoring System (ITS) to trace the learners’ knowledge, and to support downstream tasks such as recommendation, adaptive learning, and personalised feedback [32], or as a resource that can be exposed to the learner, for example through open learner models [9].

Learner models have recently come to include information about the domain as well, referred to as a domain model [30]. Concretely, the learning domain model includes learning items (exercises), knowledge components (KCs), and the relations between them [30]. While most early learner models assumed KCs to be independent, more recent work has increasingly explored structured representations of KCs and their relations, e.g. hierarchical or prerequisite relations [26]. This evolution reflects the intrinsic nature of educational domains, in which KCs are rarely isolated but are instead meaningfully connected. Incorporating domain structure naturally raises the question of propagating knowledge estimates within this structure. Since KCs are connected, evidence observed on one KC may carry implications for related KCs [36]. Existing approaches differ in how they address this propagation problem, ranging from expert-defined propagation rules to probabilistic propagation policies and, more recently, to neural network–based latent propagation [36, 20, 23]. Overall, taking domain structure into account has significantly improved the quality of knowledge models and, consequently, learner models [11].

In educational settings, learner interaction data are often sparse and unevenly distributed across KCs, leading to increased uncertainty—specifically, epistemic uncertainty—in knowledge estimates [13]. Throughout this paper, the term uncertainty is used to refer to epistemic uncertainty, i.e., uncertainty arising from limited evidence about a learner’s mastery of a KC. It reflects how strongly a mastery estimate is supported by observations, and is reducible as more learner–KC interactions are collected. Prior work suggests that quantifying uncertainty improves next-question correctness prediction by mitigating overconfident predictions, particularly when evidence is sparse [13, 25, 3]. In learner models, this uncertainty is rarely considered. Yet, some approaches quantify uncertainty when predicting the correctness of the next learner interaction [14, 25]. However, these approaches typically provide uncertainty estimates on unstructured sets of KCs.

In parallel, research on open learner models emphasises the importance of transparency and explainability in learner modeling by making aspects of the learner model explicit and consultable by the learner [9, 8]. Providing uncertainty estimates alongside mastery estimates further supports the idea of opening the learner model to the learner, as it enables her to better interpret, reflect on, and trust the information presented in the learner model.

In this context, our work aims to build a unified learner model that incorporates not only the domain structure, but also the learner’s knowledge and associated uncertainty throughout this structure. We hypothesize that estimating uncertainty throughout the domain structure can improve the quality of the learner model. We raise the following research question: RQ: Does incorporating the domain structure impact the learner model?
To answer this question, we raise two sub-research questions:
S-RQ1: How to propagate uncertainty alongside knowledge mastery within the domain structure?
S-RQ2: Under which settings does the domain structure provide the greatest benefits?

The rest of the paper is organized as follows. Section 2 reviews related work, with a particular focus on learner and domain modeling, the use of knowledge graphs in education, and mastery propagation within graph-structured representations. Section 3 presents KT-DUSL, a learner model that uses subjective logic to model knowledge mastery and uncertainty in a hierarchical domain structure. Section 4 describes the experimental setup, and Section 5 discusses the results. The code is publicly available at https://github.com/anonymousreseach/KT-DUSL.git.

2. RELATED WORK

This section reviews prior work related to our contribution. We first discuss learner and domain modeling approaches in ITSs. We then review the use of knowledge graphs in education. Finally, we examine existing approaches to knowledge mastery propagation and the quantification of uncertainty in learner models.

2.1 Learner and Domain Modeling

Learner modeling represents learners’ abilities and behavior through interaction data [30, 19]. It supports personalization and instructional decision-making in ITSs [19].

Learner models may also capture affect, motivation, metacognition, or preferences [18, 32]. However, the dominant line of work focuses on modeling learners’ mastery of knowledge concepts [16, 30]. This problem is commonly framed as knowledge tracing, which tracks latent knowledge over time to predict future performance [16, 31, 30].

Learner modeling is related to, but distinct from, domain modeling [18, 30]. Learner models encode the learner’s current and evolving knowledge with respect to a domain, while domain models specify the content structure (items, knowledge components and relations) [27]. Domain models are often treated as relatively stable, but they can be refined from data when the initial structure is imperfect [10, 29]. In contrast, learner models are inherently dynamic, as each interaction updates knowledge mastery estimates and can incorporate learning and forgetting processes [16, 30].

Prior work varies in how domain and learner models are represented and coupled [18, 30]. Many systems keep the domain structure fixed (e.g., a predefined KCs mapping) and infer learner mastery on top of it [16, 15]. Other approaches learn or exploit dependencies between KCs to propagate learner-specific evidence or mastery estimates across related KCs  [10, 26].

Beyond predictive accuracy, learner models also play an important role in explainability. In this context, research on open learner models emphasizes the importance of representations that make learner knowledge states explainable to both learners and instructors [9, 8]. Building on this principle, we adopt a learner model that explicitly represents both knowledge mastery and epistemic uncertainty within the domain structure. Such structured representations can, in principle, be exposed to learners to support reflection, self-regulated learning, and trust [9, 21]. In practice, KT-DUSL supports this by maintaining explicit, interpretable KC-level estimates of both mastery and uncertainty within the learner’s PKG, which can be visualized (e.g., as a concept-level dashboard) or queried to provide feedback on uncertain areas. This allows learners and instructors to inspect not only what is known, but also where knowledge estimates remain uncertain, thereby supporting informed decision-making and targeted learning.

2.2 Knowledge Graphs

Knowledge graphs (KGs) are a widely used formalism for representing structured knowledge, where concepts are modeled as nodes and semantic relations between them as edges  [1]. In educational systems, KGs have been extensively used to represent domain knowledge—often referred to as domain knowledge graphs, i.e., graph-based representations of domain models—in the form of concept maps, prerequisite graphs, or hierarchical concept structures [1]. Such a representation makes explicit the relationships between knowledge components and can be used to support curriculum design, content sequencing, and learner assessment [18].

In most educational applications, domain knowledge graphs are shared across learners and capture expert-defined domain structure [17]. Learner modeling is then performed on top of this structure by associating learner-specific information -such as knowledge mastery estimates- with individual knowledge components. While effective, this separation typically treats the domain structure as static and limits the extent to which learner knowledge can be jointly reasoned about and updated across the graph, as learner evidence is incorporated locally rather than being propagated through the domain structure  [34].

More recently, knowledge graphs have been extended toward learner-centered representations, giving rise to the notions of personalized and personal knowledge graphs. Personalized knowledge graphs typically adapt an existing general-purpose or domain-level KG by filtering, weighting, or annotating entities and relations according to a user’s preferences or profile  [5]. In contrast, personal knowledge graphs (PKGs) exist alongside other KGs and contain a disjoint set of entities that are of direct relevance to a single individual, while optionally linking to external knowledge graphs [33].

Different definitions of PKGs emphasize complementary aspects of the concept. From an ownership and access-control perspective, a PKG is a knowledge graph fully controlled by an individual, with the primary goal of supporting personalized services [33]. From a semantic perspective, PKGs represent structured knowledge about entities and relations of personal relevance, organized around a central user node [5].

In educational contexts, PKGs provide a natural foundation for learner-centered modeling, as they allow domain knowledge to be explicitly organized around the learner. Ain et al. [2] propose a PKG-based learner modeling approach in which learners construct their own PKGs by marking concepts they did not understand while interacting with learning items. More recently, Alatrash et al. [4] leverage PKGs in combination with graph neural networks to model learner knowledge mastery in a transparent manner, demonstrating the potential of graph-based representations for learner modeling.

Despite these advances, most PKG-based educational approaches attach learner state to graph nodes in a static or ad-hoc manner, typically as scalar mastery values or latent embeddings, offering limited support for principled propagation of learner knowledge and uncertainty across the graph.

2.3 Mastery Propagation through the Domain Structure

A substantial body of work goes beyond estimating mastery independently per KC and instead propagates learner evidence through a domain structure.

Early educational systems commonly relied on learner models coupled to concept networks, in which prerequisite or semantic links define hand-crafted propagation rules: evidence of mastery or failure on one KC is deterministically or heuristically propagated to related KCs. Such approaches rely on expert-defined rules and propagation strengths, resulting in transparent but largely fixed propagation behavior [7].

A second line of work embeds the domain structure directly into probabilistic graphical models, where KC mastery variables are treated as latent states and evidence is propagated through structured dependencies. Under tree-structured hierarchies, Gao et al. [20] propose KT\(^2\), which models concept mastery using a Hidden Markov Tree to propagate evidence along parent-child relations. Beyond trees, Bayesian networks and dynamic Bayesian networks represent more general KC topologies, allowing updates on one KC to influence related prerequisites or successors [22]. Related formulations further introduce learner-specific factors or jointly infer prerequisite structure from interaction data [28, 12].

More broadly, probabilistic approaches to learner modeling have long leveraged Bayesian networks to represent dependencies between KCs and to reason under uncertainty [15]. Millán et al. [24] provide a general framework for engineering Bayesian student models, illustrating how conditional dependencies between concepts enable evidence propagation and learner knowledge mastery updates.

Finally, some deep knowledge tracing models incorporate domain structure using graph neural networks and perform inference through message passing on KC graphs or question-KC bipartite graphs [26, 34]. GIKT further leverages graph convolution to exploit higher-order question-KC relations and mitigate data sparsity [35]. While effective in terms of predictive performance, these approaches operate on latent embeddings and generally lack explicit, interpretable KC-level mastery estimates.

Uncertainty Modeling and PropagationAlthough many learner models rely on probabilistic formulations, uncertainty is typically treated implicitly rather than explicitly represented in the learner model. As a result, mastery estimates are often reduced to point values, conflating low mastery due to consistent failure with low mastery arising from limited evidence. Moreover, most propagation policies focus on transferring mastery alone, without modeling how epistemic uncertainty should evolve across structured representations. These limitations are particularly problematic in sparse settings, motivating learner models that jointly represent and propagate both knowledge mastery and uncertainty.

3. INTEGRATING THE DOMAIN STRUCTURE IN A LEARNER MODEL

In this work, we introduce KT-DUSL, a new learner model that explicitly leverages the domain structure, and relies on a dedicated propagation policy to quantify both knowledge mastery and uncertainty throughout this structure. Before presenting the KT-DUSL model, we first recall propagation policies commonly used in the literature.

3.1 Propagation in a Domain Structure

Learner models that manage a domain structure rely on propagation policies. Most works in the literature aim at propagating knowledge mastery throughout the knowledge components (KCs) of the domain. Two general principles govern such a propagation: (i) information should be attenuated as it moves away from its source, and (ii) information originating from multiple sources should be combined coherently. These principles are commonly implemented through a discount operator and a consensus operator. We introduce these operators in a general setting. They will then be instantiated in KT-DUSL.

Discount Operator. The discount operator models the attenuation of information during propagation across the domain structure. A discount factor \(r_{i,j} \in [0,1]\) controls the strength of propagation from a KC \(c_i\) to another KC \(c_j\).

Consensus Operator. When a KC receives information from multiple related KCs, these sources of information must be aggregated in a principled manner. The consensus operator defines how multiple propagated pieces of information are combined.

3.2 The KT-DUSL Learner Model

KT-DUSL is a learner model that is designed to manage simultaneously the domain structure and the learner’s mastery and associated uncertainty throughout this structure. KT-DUSL relies on two core elements: (i) a Personal Knowledge Graph (PKG), instantiated for each learner, (ii) subjective logic to estimate knowledge mastery and associated uncertainty. The originality of KT-DUSL lies in the propagation policy used to propagate these quantities throughout the domain structure. KT-DUSL also supports explainability by maintaining explicit KC-level knowledge mastery and uncertainty estimates within the PKG, whose updates can be made visible to the learner.

From now on, the general term exercise will be used to refer to a learning item.

3.2.1 Personal Knowledge Graph Structure

KT-DUSL relies on a dedicated Personal Knowledge Graph (PKG), constructed for each learner, that jointly represents (i) the domain structure, made up of both a hierarchy of knowledge components (KCs) where higher-level KCs correspond to broader concepts and leaf-level KCs capture more fine-grained concepts, and the set of exercises linked to leaf-level KCs, (ii) learner–exercise interactions, and (iii) learner-specific estimates of knowledge mastery and uncertainty on the KCs of the domain structure.

Let \( PKG_l = (V, E) \) denote the personal knowledge graph of learner \( l \). The set of nodes is defined as \( V = C \cup X \cup \{l\} \), where \( C \) is the set of knowledge components, \( X \) is the set of exercises, and \( l \) denotes the node that represents the learner. \(E\) represents the set of edges. Each \( (c_p, c_c) \in E \) encodes hierarchical membership relations between KCs. Each exercise \( x \in X \) is associated with a set of KCs \( C(x) \subseteq C \). The learner node is connected to each exercise node through the observed interaction outcome, and to each KC through the learner’s current knowledge mastery and uncertainty estimates for that KC.

3.2.2 Subjective Logic to Estimate Mastery and Uncertainty of KCs

In the recent literature, the KT-USL learner model [3], has proposed to use subjective logic to trace learners’ knowledge mastery and uncertainty on KCs.

At time \(t\), the knowledge state of learner \(l\) on \(c_i \in C\), is represented by \(\omega _t(l,c_i)\), a subjective logic (SL) opinion:

where \(b_t(l,c_i)\), \(d_t(l,c_i)\), and \(u_t(l,c_i)\) denote belief in mastery, disbelief, and epistemic uncertainty, respectively, with \(b_t(l,c_i) + d_t(l,c_i) + u_t(l,c_i) = 1\) and \(a\) is a base-rate prior.

Opinions are grounded in accumulated interaction evidence \((r_t(l,c_i), s_t(l,c_i))\), corresponding to the number of correct and incorrect responses observed for learner \(l\) on \(c_i\). Following subjective logic, belief, disbelief, and uncertainty are computed as:

where \(W>0\) is a prior weight that controls the contribution of epistemic uncertainty under sparse evidence. The knowledge mastery estimate \(m_t(l,c_i)\) is defined as the expected value of the opinion and ranges between \(0\) and \(1\).

This formulation explicitly distinguishes between low mastery supported by substantial evidence and low mastery arising from insufficient information, while remaining local to individual (learner, concept) pairs.

It is important to note that this formulation captures epistemic uncertainty arising from limited learner–KC evidence. Other sources of uncertainty may also affect learner modeling, including noisy or incomplete exercise–KC mappings, rarely accessed concepts, or inaccuracies in the domain structure. These sources are not explicitly modeled in KT-DUSL, which assumes a fixed and reliable domain structure. Therefore, the uncertainty quantified in this work should be interpreted as evidence-based epistemic uncertainty rather than a comprehensive account of all uncertainty sources.

3.2.3 Mastery and Uncertainty Propagation in KT-DUSL

KT-DUSL extends the KT-USL model, by explicitly leveraging the domain structure. KT-DUSL relies on a propagation policy for both mastery and uncertainty across KCs of the structure. More precisely, KT-DUSL propagates information between hierarchically related KCs. This propagation allows the learner model to compensate for sparse or unevenly distributed evidence.

KT-DUSL relies on \(PKG_l\), the Personal Knowledge Graph of a learner \(l\), where the nodes of \(PKG_l\) represent the KCs of the domain, the exercises and the learner \(l\). Figure 1 represents an example of \(PKG_l\), where blue nodes represent the KCs of the domain, orange nodes represent the exercises and the pink node represents the learner. The edges of the PKG encode three types of relations: hierarchical relations between KCs, relations between exercises and leaf-level KCs and relations between the learner and the exercises.

KT-DUSL operates in an online setting. First, at time \(t\), a learner-exercise interaction generates new evidence that updates learner-exercise links (lower part of Figure 1).
Second, KT-DUSL relies on the KT-USL model [3] to update the subjective logic opinion of leaf-level KCs \(c_i\), denoted \(\omega _t(l,c_i)\), from the previous opinion \(\omega _{t-1}(l,c_i)\). Recall that from this opinion, the learner’s mastery and uncertainty on these leaf-level KCs can be inferred (see Equation 2). When a KC is associated with multiple exercises (in the case of batch processing of interactions), evidence is extracted from all associated exercises. The opinion on this KC is then calculated using this evidence.
Third, a specific propagation policy is performed across the hierarchy (relations between blue nodes in Figure 1). The main contribution of KT-DUSL lies in this propagation policy. It relies on the discount and consensus operators presented in Section 3.1. KT-DUSL instantiates these operators using subjective logic.

In KT-DUSL we propose to distinguish between two propagation directions: upward propagation, i.e., from a KC to hierarchically higher KCs, and downward propagation, i.e., from a KC to hierarchically lower KCs. Upward propagation is performed first, then downward propagation is performed. These two directions are governed by distinct discount rates, \(r^{\uparrow }\) and \(r^{\downarrow }\). In KT-DUSL an opinion on \(c_i\) can be propagated only from \(c_i\) to a directly related KC \(c_j\). This opinion is first discounted as follows:

where \(r_{i,j}\) denotes the discount rate associated with the hierarchical relation from \(c_i\) to \(c_j\) (\(r_{i,j}\) takes one of the two values among \(r^{\uparrow }\) and \(r^{\downarrow }\)), and controls how much belief and disbelief are transferred, with the remaining mass assigned to epistemic uncertainty.

Considering recipient KCs, a KC \(c_i\) may receive multiple discounted opinions. In KT-DUSL, these opinions come from its related KCs in the hierarchy. Let \(\Omega _t(l,c_i)\) denote the set of all opinions propagated to \(c_i\) at time \(t\). Second, the opinion is updated using the subjective logic consensus operator:

The choice of this consensus operator comes from its ability to weight the contribution of different opinions according to their uncertainty. It ensures that opinions supported by stronger evidence exert greater influence on \(c_i\).

To summarize, after each learner-exercise interaction, KT-DUSL performs: (i) a local update of the opinions of the leaf-level KCs associated with the exercise of the newly observed evidence using KT-USL; and (ii) a propagation step that diffuses mastery and uncertainty throughout the domain hierarchy (up and down). Propagation relies on both discount and consensus operators. As a result, the learner’s PKG evolves over time to reflect both accumulated evidence and structural relations between concepts.

4. EXPERIMENTAL SETUP

4.1 Datasets

To evaluate KT-DUSL, we perform experiments on two datasets from the literature. Both datasets are related to mathematics.

Junyi. The Junyi dataset is derived from an online educational platform that provides large-scale logs of student interactions with mathematical questions.¹

EEDI (Task 1-2). EEDI is an online mathematics learning platform that provides interaction logs together with rich metadata describing questions, learners, and answers.²

Table 1 summarizes the main characteristics of the two datasets. A key structural difference between them has motivated their choice: in Junyi, each exercise is associated with a single KC, whereas in EEDI each exercise is associated to multiple KCs, with an average of 4.17 concepts per question. As a result, EEDI is denser than Junyi with on average about twice as many interactions per concept. We investigate if these differences affect the impact on the mastery and uncertainty, as well as the performance on downstream tasks.

4.2 Metrics

We quantify the impact of the dedicated domain-informed propagation on the learner model. Concretely, we measure the absolute change in knowledge mastery and uncertainty, induced by propagation on leaf-level KCs, defined as \( |\Delta | = |x_{\mathrm {with\;domain}} - x_{\mathrm {without\;domain}}| \). Absolute deltas are reported for mastery (KT-A \(\rightarrow \) KT-DA and KT-USL \(\rightarrow \) KT-DUSL) and for uncertainty (KT-USL \(\rightarrow \) KT-DUSL).

In line with the literature, we also evaluate KT-DUSL on the traditional next-question correctness prediction task, using the following standard metrics:

• AUC (Area Under the ROC Curve), which measures the quality of probabilistic ranking independent of a fixed decision threshold.
• Accuracy@0.5, computed by thresholding predicted correctness probabilities at 0.5.
• LogLoss, which assesses the calibration and sharpness of predicted probabilities. The lower, the better.

4.3 Baseline Models

To address the RQ raised in the introduction, we empirically evaluate KT-DUSL, and compare it against a set of baseline models. This evaluation will contribute to quantifying the effects of both uncertainty modeling and propagation policy.

KT-A is a deterministic baseline model that estimates learner mastery for leaf-level KCs by averaging observed correctness outcomes. It does not quantify uncertainty and does not exploit domain structure.

KT-DA extends KT-A by propagating average mastery estimates across the KC hierarchy using mean-based instantiations of the discount and consensus operators.

KT-USL (see Section 3.2.2) models learner mastery for leaf-level KCs only, and quantifies uncertainty using subjective logic. KT-USL treats each KC independently, so does not propagate information across the domain structure.

4.3.1 Parameter Initialization and Control

KT-DUSL introduces parameters governing both propagation and subjective logic. The coefficients \(r^{\uparrow }\) and \(r^{\downarrow }\) control the strength of upward and downward propagation in the domain hierarchy, while the subjective logic formulation relies on the base rate \(a\) and the non-informative prior weight \(W\). Although several parameter settings were explored on a validation set, we report results for fixed values due to space constraints: \(r^{\uparrow } = r^{\downarrow } = 0.8\), \(a = 0.5\), and \(W = 2.0\), which corresponds to the best-performing configuration.

5. EVALUATION OF THE IMPACT OF THE DOMAIN STRUCTURE

In this section we evaluate the impact of the domain structure on the learner model, and on the performance of a downstream task, including in high uncertainty settings.

5.1 Impact on the Learner Model

We start by analyzing the impact of domain-informed propagation on both knowledge mastery and uncertainty. Table 2 reports the median and quartiles of the absolute changes induced across different datasets. Recall that Junyi and EEDI mainly differ in the average number of KCs per exercise. To make the datasets more comparable, we construct a third dataset, which is a mono-concept version of the EEDI dataset, by retaining a single leaf concept per exercise. This concept is selected deterministically by first choosing the deepest associated KC in the hierarchy (i.e., the most specific concept), and mapping it to a leaf node when necessary.

Let us first consider the impact of integrating the domain structure, on the knowledge mastery. Recall that knowledge mastery ranges between 0 and 1, hence also the impact. Integrating the domain structure in a model that does not manage uncertainty (KT-DA vs. KT-A), has a significant impact on the knowledge mastery in a mono-concept setting, with a median of impact equal to 0.107. On EEDI, Q3 is equal to 0.204, which means that the impact is larger than 20 points of percentage for 25% of the concepts. Yet, this impact is twice lower in a multi-concept setting. Managing domain structure on uncertainty-aware learner models (KT-DUSL vs. KT-USL) impacts the knowledge mastery to a lower extent (median around 0.03 on all datasets).

Focusing on the change in uncertainty on the KCs, the impact is larger on EEDI (both settings) than on Junyi. The highest impact is obtained on the multi-concept setting (median equal to 0.124 and Q3 is equal to 0.241). Recall that this large impact on uncertainty occurs in a context of small impact on knowledge mastery.

We can conclude that on a mono-concept setting, KT-DA impacts significantly more the knowledge mastery than KT-DUSL. Besides, KT-DUSL impacts more uncertainty in multi-concept settings.

Based on these conclusions we wonder if these significant changes in knowledge mastery and uncertainty translate into improved performance on a downstream task.

5.2 Impact on the Next-Question Correctness Prediction Task

In this section, we aim at evaluating the impact of the domain structure on a downstream task. In particular, we choose to evaluate on the next-question correctness prediction task.

5.2.1 Overall Impact

To evaluate the overall impact of the domain structure on the next-question correctness prediction task, we use the three metrics introduced in section 4.2, namely AUC, ACC@0.5 and LogLoss.

We first consider Junyi dataset. Evaluation metrics values are displayed in Table 3.

The impact of the exploitation of the domain structure can be evaluated by comparing two pairs of models. First, we compare models that manage KC knowledge mastery only, concretely KT-A and KT-DA. The domain structure has no effect on AUC or accuracy but improves LogLoss significantly. The effect on AUC is thus not related to the significant impact on knowledge mastery (see Section 5.1). Second, we compare models that quantify and manage uncertainty, concretely KT-USL and KT-DUSL. KT-DUSL achieves slightly higher performance, particularly in AUC (0.705 to 0.710) and LogLoss (0.556 to 0.551). Here again, the impact on knowledge mastery was null. Overall, we can say that exploiting the domain structure and propagating mastery and uncertainty through this structure remains beneficial when used with an uncertainty-aware model, although the impact on the learner model is limited.

Let us now examine whether these conclusions hold on the EEDI dataset, in which each exercise is associated with multiple KCs. Performance is displayed in Table 4.

On this dataset, the domain structure does not consistently improve these metrics. Indeed, KT-DUSL does not improve AUC and accuracy, and only slightly improves LogLoss. We hypothesize that this is due to the multi-concept setting, in which much of the relational information between concepts is implicitly captured through the exercise-KC associations. As a consequence, the explicit relation provided by the domain structure is already present in the data. This is in line with the limited impact on knowledge mastery from the previous section.

To test this hypothesis, we perform similar evaluation on the mono-concept version of the EEDI dataset, The associated performance is reported in Table 5.

We can see that, as on Junyi, considering the domain structure improves performance. KT-DUSL consistently outperforms all baseline models across AUC, ACC@0.50, and LogLoss, with particularly large gains over KT-USL (AUC 0.680 to 0.702; LogLoss 0.616 to 0.601). This tends to show that, in the context of mono-concept datasets, that can be associated to sparse data, the domain structure contributes to the quality of the learner model. More precisely, the quality of downstream next-question correctness prediction task is increased. This thus supports our hypothesis.

These experiments highlight two important conclusions: 1) quantifying uncertainty is broadly beneficial across datasets, 2) considering the domain structure has a context-dependent impact. If the relations between KCs are already implicitly encoded in the dataset (such as multiple KCs per exercise), domain-informed propagation does not provide additional information, so does not improve the performance. In contrast, in the context of sparse data (single concept per exercise), the domain structure contributes to improve the performance.

These findings lead us to investigate deeper the relation between the amount of data available, the uncertainty and performance. Concretely, we wonder if 1) the domain structure is more beneficial in the context of limited information about the learner, 2) the domain structure is more beneficial in high uncertainty settings.

5.2.2 Limited Evidence Settings

In this section, we analyze the benefits of incorporating the domain structure under different levels of available evidence per learner-concept pair by comparing KT-USL and KT-DUSL. The amount of evidence is quantified by \(k\), the number of past interactions a learner had with a given KC. Concretely, \(k\) denotes the interaction index for a learner-KC pair: \(k=1\) corresponds to the learner’s first encounter with the KC, \(k=2\) to the second interaction, and so on.

Table 6 reports AUC, ACC@0.5 and LogLoss for different values of \(k\). Higher values of \(k\) correspond to smaller dataset sizes. We can see that KT-DUSL consistently outperforms KT-USL on the three metrics, whatever is the value of \(k\). The largest gains are observed with low values of \(k\), i.e. with little evidence.

For \(k=1\), incorporating domain structure yields a substantial improvement in AUC (11% improvement, from 0.500 to 0.557) and a notable reduction in LogLoss (5.3% improvement, from 0.693 to 0.656). This indicates that when little information about the learner is available, better discrimination and probability calibration are reached. As evidence accumulates (\(k=3\) and \(k=5\)), the performance gap narrows but remains consistent, suggesting that the domain structure continues to provide complementary information beyond local evidence. For larger values of \(k\) (\(k \geq 10\)), difference in performance between the two models is almost null, reflecting the fact that the structure becomes marginal as evidence increases.

Overall, these results indicate that domain-informed propagation is most beneficial in the context of low evidence, where it contributes to effectively compensate for sparse learner–KC interactions. Its impact attenuates as evidence increases.

Let us now study to what extent these conclusions hold on EEDI dataset (mono-concept and multi-concept). Table 7 shows performance on the mono-concept EEDI dataset. As on Junyi, domain-informed propagation is consistently beneficial across all evidence levels. For small values of \(k\), KT-DUSL yields large gains over KT-USL, particularly in AUC (19% improvement, from 0.500 to 0.594 for \(k=1\)) and LogLoss (4% improvement, from 0.693 to 0.666). As on Junyi, as \(k\) increases, improvements decrease for all metrics. This confirms the importance of domain structure when evidence is low and becomes useless when evidence increases.

On the multi-concept EEDI dataset, we observe a markedly different behavior. The benefits of domain-informed propagation are almost null, whatever is the amount of evidence. We choose to not show values here. This confirms again the hypothesis drawn in section 5.2.1, the multi-concept setting implicitly encodes the domain structure, making the explicit domain structure useless. This is true even when evidence is low.

5.2.3 High Uncertainty Settings

We now intend to evaluate the benefits of incorporating the domain structure considering different uncertainty values. To this aim, we evaluate the impact of uncertainty on predictive performance, and in particular on AUC. Figures 2 and  3 show the AUC computed over sliding windows of 200,000 interactions, where each point corresponds to the average uncertainty of leaf KCs within a window, for the Junyi and mono-concept EEDI datasets. In both datasets, performance is consistently lower with high uncertainty values (30% decrease when uncertainty equals 1.0 compared to the case where uncertainty equals 0.0). This reflects the intrinsic difficulty of the prediction task in high uncertainty settings. However, in both datasets, KT-DUSL outperforms KT-USL when uncertainty is larger than 0.3. The higher the uncertainty, the larger the increase. When uncertainty is maximal (1.0), this increase reaches 12% and 18% on Junyi and EEDI respectively. This confirms the relevance of managing domain structure in uncertainty-aware models in the context of high uncertainty.

The previous two sections have highlighted the fact that the domain structure contributes to improve the performance on the next-question correctness prediction downstream task. This effect is particularly pronounced in low-evidence and high-uncertainty settings, where the domain structure contributes the most. The strong relation between evidence and uncertainty values is clearly stated thanks to these experiments.

These results allow us to answer the research questions introduced in Section 1. Incorporating the domain structure actually impacts both the learner model and downstream prediction performance, thereby addressing S-RQ1. In particular, domain-informed propagation is most beneficial in settings characterized by high uncertainty or limited evidence, while its contribution becomes marginal when sufficient learner-KC information is available, thus addressing S-RQ2.

6. CONCLUSION

In this work, we studied the relevance of managing domain structure in learner models and associated downstream tasks. To this aim, we introduced KT-DUSL, a domain-informed learner model that jointly represents and propagates knowledge mastery and associated uncertainty in a learner model. Building on subjective logic, KT-DUSL performs local evidence updates and then propagates the resulting opinions through the domain structure, using discounting and consensus operators, thereby preserving uncertainty throughout the process.

Experiments on two mathematics-related datasets: Junyi and EEDI show that explicit uncertainty modeling consistently improves next-question correctness prediction, especially in AUC and LogLoss. We further find that, in the context of mono-concept exercises, domain-informed propagation provides the largest gains in high-uncertainty and low evidence per learner-concept pair settings. In contrast, in the multi-concept EEDI setting, hierarchical propagation brings limited benefit, suggesting that rich exercise-concept information encodes substantial relational information, and provides as much information as the domain structure.

Future work will leverage the joint propagation of knowledge mastery and epistemic uncertainty to develop an adaptive learning system that is guided by both estimated mastery and uncertainty within the domain structure. We also plan to investigate privacy-preserving deployments, in particular through federated learning, where learners’ personal knowledge graphs are maintained locally and only a limited set of shared parameters is exchanged, thereby enabling collaborative learning.

References

1. H. Abu-Rasheed, M. Dornhöfer, C. Weber, G. Kismihók, U. Buchmann, and M. Fathi. Building contextual knowledge graphs for personalized learning recommendations using text mining and semantic graph completion. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT), pages 36–40. IEEE, 2023.
2. Q. U. Ain, M. A. Chatti, P. A. Meteng Kamdem, R. Alatrash, S. Joarder, and C. Siepmann. Learner modeling and recommendation of learning resources using personal knowledge graphs. In Proceedings of the 14th learning analytics and knowledge conference, pages 273–283, 2024.
3. R. A. Ait Chabane Rania, Brun Armelle. Uncertainty-aware knowledge tracing: Towards the use of subjective logic. In XAI-Ed 2026 Proceedings of International Conference on Learning Analytics Knowledge (LAK 2026), 2026. To be published.
4. R. Alatrash, M. A. Chatti, Q. U. Ain, and S. Joarder. Transparent learner knowledge state modeling using personal knowledge graphs and graph neural networks. In Adjunct proceedings of the 32nd ACM conference on user Modeling, adaptation and personalization, pages 591–596, 2024.
5. K. Balog and T. Kenter. Personal knowledge graphs: A research agenda. In Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, pages 217–220, 2019.
6. F. Böck, M. Ochs, A. Henrich, D. Landes, J. L. Leidner, and Y. Sedelmaier. Learner models: design, components, structure, and modelling; a systematic literature review. 2025.
7. P. Brusilovsky and E. Millán. User models for adaptive hypermedia and adaptive educational systems. In The adaptive web: methods and strategies of web personalization, pages 3–53. Springer, 2007.
8. S. Bull. There are open learner models about! IEEE Transactions on Learning Technologies, 13(2):425–448, 2020.
9. S. Bull and J. Kay. Open learner models. In Advances in intelligent tutoring systems, pages 301–322. Springer, 2010.
10. H. Cen, K. Koedinger, and B. Junker. Learning factors analysis–a general method for cognitive model evaluation and improvement. In International conference on intelligent tutoring systems, pages 164–175. Springer, 2006.
11. P. Chen, Y. Lu, V. W. Zheng, and Y. Pian. Prerequisite-driven deep knowledge tracing. In 2018 IEEE international conference on data mining (ICDM), pages 39–48. IEEE, 2018.
12. Y. Chen, J. P. González-Brenes, and J. Tian. Joint discovery of skill prerequisite graphs and student models. International Educational Data Mining Society, 2016.
13. W. Cheng, H. Du, C. Li, E. Ni, L. Tan, T. Xu, and Y. Ni. Uncertainty-aware knowledge tracing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27905–27913, 2025.
14. S. T. Christie, C. Cook, and A. N. Rafferty. Uncertainty-preserving deep knowledge tracing with state-space models. arXiv preprint arXiv:2407.17427, 2024.
15. C. Conati, A. Gertner, and K. Vanlehn. Using bayesian networks to manage uncertainty in student modeling. User modeling and user-adapted interaction, 12(4):371–417, 2002.
16. A. T. Corbett and J. R. Anderson. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4):253–278, 1994.
17. F.-R. Dang, Y. Liu, W. Zhao, and W. Zhang. Constructing an educational knowledge graph with concepts linked to wikipedia. Journal of Computer Science and Technology, 36(5):1200–1211, 2021.
18. M. C. Desmarais and R. S. d. Baker. A review of recent advances in learner and skill modeling in intelligent learning environments. User Modeling and User-Adapted Interaction, 22(1):9–38, 2012.
19. H. Drachsler, K. Verbert, O. C. Santos, and N. Manouselis. Panorama of recommender systems to support learning. In Recommender systems handbook, pages 421–451. Springer, 2015.
20. X. Gao et al. A hierarchical probabilistic framework for incremental knowledge tracing in classroom settings. arXiv preprint arXiv:2506.09393, 2025.
21. D. Hooshyar, M. Pedaste, K. Saks, Ä. Leijen, E. Bardone, and M. Wang. Open learner models in supporting self-regulated learning in higher education: A systematic literature review. Computers & education, 154:103878, 2020.
22. T. Käser, S. Klingler, A. G. Schwing, and M. Gross. Dynamic bayesian networks for student modeling. IEEE Transactions on Learning Technologies, 10(4):450–462, 2017.
23. P. Lv, X. Wang, J. Xu, and J. Wang. Intelligent personalised exercise recommendation: A weighted knowledge graph-based approach. Computer Applications in Engineering Education, 29(5):1403–1419, 2021.
24. E. Millán, T. Loboda, and J. L. Pérez-De-La-Cruz. Bayesian networks for student model engineering. Computers & Education, 55(4):1663–1683, 2010.
25. J. Mitton, P. Bhattacharyya, R. Abboud, and S. Woodhead. Uncertainty-aware knowledge tracing models. arXiv preprint arXiv:2509.21514, 2025.
26. H. Nakagawa, Y. Iwasawa, and Y. Matsuo. Graph-based knowledge tracing: modeling student proficiency using graph neural network. In IEEE/WIC/aCM international conference on web intelligence, pages 156–163, 2019.
27. D. Oblinger and J. K. Lippincott. Learning spaces, volume 444. Educause Washington, DC, 2006.
28. Z. A. Pardos and N. T. Heffernan. Modeling individualization in a bayesian networks implementation of knowledge tracing. In International conference on user modeling, adaptation, and personalization, pages 255–266. Springer, 2010.
29. P. I. Pavlik Jr, H. Cen, and K. R. Koedinger. Performance factors analysis–a new alternative to knowledge tracing. Online submission, 2009.
30. R. Pelánek. Bayesian knowledge tracing, logistic models, and beyond: An overview of learner modeling techniques. User Modeling and User-Adapted Interaction, 27(3):313–350, 2017.
31. C. Piech, J. Bassen, J. Huang, S. Ganguli, M. Sahami, L. J. Guibas, and J. Sohl-Dickstein. Deep knowledge tracing. Advances in neural information processing systems, 28, 2015.
32. J. Rihák and R. Pelánek. Choosing a student model for a real world application. In Proceedings of the Workshop on Building ITS Bridges Across Frontiers, 2016. ITS Workshop.
33. M. G. Skjæveland, K. Balog, N. Bernard, W. Łajewska, and T. Linjordet. An ecosystem for personal knowledge graphs: A survey and research roadmap. AI Open, 5:55–69, 2024.
34. S. Tong, Q. Liu, W. Huang, Z. Hunag, E. Chen, C. Liu, H. Ma, and S. Wang. Structure-based knowledge tracing: An influence propagation view. In 2020 IEEE international conference on data mining (ICDM), pages 541–550. IEEE, 2020.
35. Y. Yang, J. Shen, Y. Qu, Y. Liu, K. Wang, Y. Zhu, W. Zhang, and Y. Yu. Gikt: a graph-based interaction model for knowledge tracing. In Joint European conference on machine learning and knowledge discovery in databases, pages 299–315. Springer, 2020.
36. L.-Y. Zhou and Y.-Y. Wang. Simulation of personalized english learning path recommendation system based on knowledge graph and deep reinforcement learning. Scientific Reports, 15(1):34554, 2025.

© 2026 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.