ABSTRACT

Learning to program involves repeated trial-and-error attempts, leading to substantial heterogeneity in learning behavior across student cohorts and course offerings. Code-aware knowledge tracing (KT) models aim to capture how students’ programming knowledge evolves over time. However, most existing KT studies rely on data from individual courses, where student interactions are often sparse and insufficient for modeling robust performance patterns. Centralizing student data across classrooms or institutions to address this limitation is often constrained by the risk of privacy breach of sensitive code submission data, limiting the impact of existing KT methods. Federated learning (FL) provides a privacy-preserving alternative by enabling collaborative model training without sharing raw data, and recent work has shown that model personalization can further improve federated KT performance. However, the interaction between model personalization and code representation has not been systematically studied. In this paper, we present a comparative analysis of code-aware KT representations under personalized federated learning (pFL) framework, using two semesters of CodeWorkout data treated as separate clients. Our results show that personalization consistently improves predictive accuracy under federated training, but the magnitude and consistency of these improvements depend on how student code is represented. In particular, code representations that capture richer structural information and syntax-aware encodings exhibit more reliable performance gains and promise more stable learning trajectories across assignments and semesters, especially for more complex programming tasks. These findings provide practical insights for EDM researchers and practitioners on how representation choice shapes the effectiveness of personalization when deploying privacy-preserving knowledge tracing models in programming education.

Keywords

1. INTRODUCTION

With the growing reliance on learning management systems (LMSs) for the organization and delivery of instructional activities and assessments, time-ordered logs of student interactions with the digital learning environment have become available at scale. Learning analytics leverages these records to track student learning over time, and predict future performance. One prominent approach, known as knowledge tracing (KT) [37], estimates students’ latent knowledge states over time and has long been used to uncover learning trajectories, support practice recommendations, and inform adaptive instructional interventions.

Due to their data-driven nature, KT methods benefit from larger and more diverse student interaction datasets, where aggregating student data across courses and semesters enables models to capture nuanced learning behaviors and their evolution over time. However, while such data centralization/pooling can improve model performance through leveraging diverse data, it raises significant privacy concerns. Specifically, regulations like the family educational rights and privacy act (FERPA) [1] restrict student data sharing across institutions to prevent the disclosure of sensitive information. At the same time, while individual/localized training of KT models within a single course avoids these privacy concerns, it often leads to less than optimal performance due to limited data scale and diversity. This limitation is further exacerbated by the nature of student interaction data itself, which is often sparse and unevenly distributed across assignments: some problems receive only a small number of attempts or exhibit limited behavioral variation, constraining the model’s ability to learn robust representations of student knowledge and performance, particularly for less frequently attempted or more complex tasks.

Federated learning (FL) has gained traction as a promising distributed machine learning (ML) paradigm to address these challenges [24, 17]. FL allows collaborative model training among multiple data-owning entities, namely clients, without requiring the exchange of raw data. Instead, clients train models locally and share only model/gradient parameters with a central server, which aggregates them to form a global model that is redistributed for subsequent training rounds. As a result, by design, FL preserves data privacy while allowing models to benefit from knowledge learned from distributed data sources. In practice, however, data sourced from multiple clients often exhibit substantial heterogeneity - particularly evident in educational domains due to differences in cohort characteristics, instructional methods, or infrastructural contexts. Fairness and correctness considerations being central concerns in education, therefore, makes such one-size-fits-all models less effective. To address this issue, recent work has explored personalization strategies in FL, which adapt the global model to client-specific data and have been shown to improve both predictive performance and fairness. As a result, FL with personalization has attracted growing attention in educational applications [3, 33, 35, 14, 21, 13, 22, 43, 39], particularly for tasks such as KT [41, 5, 6].

Yet, programming education introduces unique challenges for KT in FL or simply, federated KT. Specifically, as learning happens through iterative code submissions to open-ended programming problems, student-generated code serves as the primary sequential evidence of learning progress, making code representation a central factor in modeling student knowledge. To cater to such distinct needs in programming education, KT models are, therefore, often code-aware, incorporating representations of student code submissions to better capture learning processes. However, while personalized federated KT has been studied, the interaction between code representation choices and personalization of federated models remains underexplored. In particular, it is unclear how different code-aware KT models behave under federated training, and how personalization impacts representation-specific strengths and weaknesses of the model. To address this gap, this paper investigates how code representation shapes the behavior and performance of code-aware KT models trained under personalized federated learning (pFL).

1.1 Research Questions

We extend three code-aware KT models: Code-DKT [38], SANN-DKT [12], and srcML-DKT [27], to FL settings to enable privacy-preserving performance prediction across semesters. To account for cross-semester heterogeneity, we incorporate pFL strategies and empirically analyze their effects across different code representations. To our knowledge, this work is the first study that employs pFL on code representation to provide data privacy and better predictive modeling for code-aware KT methods. The summary of our contributions lies in the answers to the following questions:

RQ1: Do relative performance differences arising from code representation choices in KT models persist under privacy-preserving pFL?

RQ2: To what extent does the personalization aspect of pFL impact cross-semester prediction accuracy in code-aware KT?

2. RELATED WORK

2.1 Modeling Student Learning with KT

Prior work has shown that students engage with programming practice systems in distinct ways, exhibiting varied behavioral patterns such as exploration and persistence [30]. At the same time, programming learning involves mastering a set of underlying skills or knowledge components (KCs) that are reflected across exercises and instructional materials [40]. Understanding how these behaviors relate to the acquisition and application of knowledge over time has been a central focus of learning analytics [37], motivating the development of knowledge tracing (KT) models. Early KT models were probabilistic, most notably Bayesian Knowledge Tracing (BKT) [8], which represents learning as a hidden Markov process and estimates the probability that a student has mastered a skill as they engage with successive tasks. Extensions of BKT introduced individualized parameters to account for variation in prior knowledge and learning rates across students [44]. Related approaches, such as Performance Factors Analysis (PFA) [28] and Learning Factors Analysis (LFA) [4], used logistic regression over prior successes and failures to estimate knowledge states. While influential, these models struggle to capture complex learning dynamics involving multiple interdependent skills and individualized learning trajectories.

The introduction of deep learning marked a major shift in knowledge tracing research. Deep Knowledge Tracing (DKT) [29] showed that recurrent neural networks can capture temporal patterns in student behavior better, leading to notable gains in predictive accuracy. Subsequent work built on this idea by incorporating memory mechanisms, attention, and richer representations. Examples include Dynamic Key-Value Memory Networks (DKVMN) [47] and Self-Attentive Knowledge Tracing (SAKT) [26], as well as later models that combine multiple architectural components to improve scalability and performance [32, 31, 18]. More recent approaches have further expanded the KT landscape by introducing relational and contrastive perspectives, such as graph-based and problem-solving–oriented models [25, 42, 16, 48]. Together, these advances have substantially improved the modeling of student learning behavior.

2.2 Potential of pFL in Code-aware KT

While KT has been effective in many educational settings, open-ended programming courses pose distinct challenges. In such courses, learners often submit multiple iterations of source code, and correctness alone provides only a partial signal of learning. Each attempt contains structural and semantic details that reflects how students reason about problems, debug errors, and refine their solutions over time. The emergence of code-aware KT models prompted moving beyond just correctness labels by explicitly modeling student code. Code-DKT uses transformed code embeddings to represent semantic details of student submissions [38], while SANN-DKT applies attention mechanism to emphasize code parts that contribute most to prediction [12]. srcML-DKT takes this further by allowing incomplete or buggy codeblocks to incorporate syntactic and semantic evolution of student code [27]. Collectively, these studies demonstrate that incorporating structural, semantic, and contextual information from code leads to a more nuanced and fine-grained understanding of programming knowledge acquisition. Additional work has explored alternative representations and objectives, including error-based features for capturing misconceptions (Error-DKT)[36], explainable KT models providing cognitive feedback (HELP-DKT)[23], methods that improve interpretability through automatically generated knowledge components (KCGen-KT)[9] or knowledge graphs (KGNN-KT)[45], and question-centered approaches that incorporate student-generated questions to support generalization (SQKT)[19].

As efforts to improve KT in programming education continue, they also reveal a key limitation: code-aware KT models typically require large and diverse datasets to perform reliably. In practice, data from a single course or semester are often insufficient, motivating the aggregation of student interaction and code data across offerings or institutions. However, such data centralization/pooling, especially raw code submissions, raises serious privacy concerns and is frequently restricted by institutional policies and regulations. FL has therefore emerged as a promising alternative, enabling collaborative model training without sharing raw data [24]. Prior work has explored federated KT in domain-generic contexts, showing that pFL can improve performance and fairness across heterogeneous clients [41, 5, 6]. However, the interaction between domain-specific code representation and pFL remains largely understudied. Specifically, existing federated KT methods do not examine how different code-aware representations behave under federated training, nor whether personalization impacts their accuracy. To address this gap, our work systematically studies how different code-aware KT representations respond to pFL, providing insights into predictive performance across decentralized programming education settings.

3. METHODOLOGY

3.1 Datasets

In this study, we will analyze two datasets from CodeWorkout, an online platform to practice Java programming.¹ The platform allows instructors to design and deliver programming courses, while providing students with the opportunity to complete exercises and receive immediate feedback [10]. CodeWorkout records detailed logs of student submissions, with scores assigned based on the number of test cases passed. Students may submit multiple solutions for the same problem, allowing us to get chronological updates of their iterative problem-solving processes. The evaluation of student knowledge in these datasets spans a range of programming concepts, including method declarations, loops, arrays, arithmetic operations, logical expressions, data types, conditionals, and nested conditional statements.

Upon its use in an introductory programming (CS1) course at a public university in the United States during the Spring 2019 and Fall 2019 semesters, two datasets were formed containing anonymized submissions for 50 programming problems across five assignments caughtSpeeding, redTicket, countCode, sum13, and canBalance) and no accompanying demographic information. ² We obtained these publicly available datasets from DataShop [20]. Each submission was graded on a continuous scale from 0 to 1 based on the proportion of test cases passed, and for modeling purposes, we categorized the outcomes into binary groups: correct (scored 1 out of 1) and incorrect submissions (failing at least one test case resulted in a 0). Table 1 shows details of the Spring 2019 and Fall 2019 datasets. We split each dataset into training, validation, and test sets in a stratified manner to account for class imbalance, using a 80:10:10 ratio.

3.2 KT Formulation

For knowledge tracing, we use deep knowledge tracing (DKT), a well-established model that along with its variants has performed reasonably well when applied on the CodeWorkout dataset [38]. In DKT formulation, a student’s learning history is represented as a sequence of attempts \(\{x_1, x_2, \ldots , x_T\}\), where each attempt \(x_t\) at time step \(t\) corresponds to a problem-correctness pair \((q_t, a_t)\). Here, \(q_t \in \{1, 2, 3, \ldots , M\}\) denotes the problem identifier for \(M\) total problems and \(a_t\) can be \(0\) or \(1\), indicating incorrectness and correctness, respectively. Each interaction is one-hot encoded as a binary vector \(x_t \in \{0,1\}^{2M}\), where \(x_{q_t + M(1 - a_t)}\) is considered \(1\), and the rest of the entries are set to \(0\). In code-aware DKT methods, each interaction is additionally associated with a code representation \(z_t\), extracted from the student’s submitted programs. They are further transformed into vectorized code representations \(\{z_1, z_2, \ldots , z_T\}\), which are concatenated with interaction embedding to form an augmented input vector \([x_t;z_t]\), as shown in Figure 1. These inputs are processed sequentially by a long short-term memory (LSTM) model that maintains an internal memory state to capture both short- and long-term temporal dependencies across attempts. The LSTM hidden state provides a latent representation of the student’s knowledge and is updated after each interaction. Starting with \(h_0\) that represents the model’s belief about the student’s knowledge before seeing any data, \(h_t\) tracks the evolving knowledge of the student across time \(t\), which can be expressed as \(h_t = \mathrm {LSTM}([x_t; z_t], h_{t-1}),\) where \(x_t = \mathrm {embedding}(q_t, a_t)\). At each time step \(t\), the model outputs a vector of predicted correctness probabilities over all problems \(\{y_1, y_2, \ldots , y_T\}\), where each entry denotes the predicted likelihood that the student will answer the corresponding problem correctly. The predicted correctness for the next interaction, \(a_{t+1}\), is obtained by selecting the entry of \(y_t\) associated with the next attempted problem \(q_{t+1}\), which is used for performance prediction.

3.3 FL Framework

The vanilla Federated Averaging (FedAvg) algorithm [24], combined separately with each of the three KT techniques, is used as the set of baseline methods in this work. This architecture is presented within the blue box in Figure 2, where, data from academic semesters are treated as belonging to distinct clients, reflecting a realistic scenario, where student interaction data are sourced from different course offerings.

A central server coordinates the federated training process across these clients through iterative rounds of local training and global aggregation. Let \(K\) denote the number of clients (semesters), and let \(\mathcal {D}_k\) represent the dataset associated with client \(k\), with size \(n_k = |\mathcal {D}_k|\). To start each global aggregation \(\tau \), the server broadcasts the current global model parameters \(\theta ^{(\tau )}\) to all clients. Each client initializes its local model with these parameters \(\theta _k^{(\tau ),0}\leftarrow \theta ^{(\tau )}\) and performs local stochastic gradient descent (SGD) updates on mini-batches sampled from its own dataset. Specifically, during each global aggregation \(\tau \), client \(k\) updates its local parameters by minimizing the empirical loss function \(\ell \) for \(e_{\max }\) local SGD steps, where the final local model is:

where \(\alpha \) denotes the learning rate and \(\mathcal {B}^{(\tau ),i}_{k}\) represents a mini-batch of size \(B\) sampled from client \(k\)’s data at SGD iteration \(i\). Subsequently, the client sends \(\theta _k^{(\tau +1)}\) to the server, and the server aggregates the local models using weighted averaging to obtain the next global model as follows:

The resulting global model \(\theta ^{(\tau +1)}\) is then reshared with all clients to initiate the next training round. In our experiments, this federated optimization process is repeated for a total of \(\mathcal {T} = 10\) global rounds (i.e., \(\tau \in \{0,\cdots ,9\}\)). All remaining FL hyperparameters are selected via grid search, with learning rates \(\alpha \in \{0.001, 0.005\}\), mini-batch sizes \(B \in \{128, 256\}\), and the number of local training steps \(e_{\max } \in \{20, 40\}\).

We next apply this FL framework to three code-aware KT models, yielding the novel FedCode-DKT, FedSANN-DKT, and FedsrcML-DKT models, which are discussed below.

3.3.1 FedCode-DKT

Code-DKT builds upon the original DKT framework by augmenting the problem-correctness interaction \(x_t\) with a code representation \(z_t\) derived from the student’s attempt logged, enabling the model to leverage information beyond binary correctness [38]. As illustrated in Figure 2(a), each code submission at time \(t\) is transformed into an abstract syntax tree (AST), which serves as the basis for extracting the corresponding code representation \(z_t\). Code-DKT employs Code2Vec [2], a neural model embeds leaf-to-leaf paths in the AST, to transform ASTs into embedded vectors ready to be used for any ML model. Code2Vec is originally designed to predict Java method names, and its internal representations can be retrieved as code embeddings, providing a compact vectorized representation of the underlying program structure. Using separate node and path embedding layers, embeddings for each of the paths are generated as vectors. These vectors are then fed into the attention layer, where the correctness score \(x_t\) determines which path is important for the final prediction. Following softmax activation and concatenation of weighted sum, the aggregated vector of correctness probabilities is generated as the output \(z_t\) of the linear layer. Once the vectors are formed for the entire dataset for reported timestamps \(1, 2, 3, \ldots , T\), they are fed as inputs to respective LSTM models (identified by semesters) along with the correctness scores for all \(T\) attempts. Each semester is locally trained with their respective data, where the global model is updated with the aggregated local model parameters. After a back-and-forth transfer for \(10\) iterations, the federated model outputs prediction probabilities.

3.3.2 FedSANN-DKT

SANN-DKT introduces a self-attention-based code representation strategy to highlight critical portions of the source code that are most indicative of future performance [12]. It employs subtree-based attention neural network (SANN) [11] instead of ASTs, as demonstrated in Figure 2(b). Under the SANN framework, a single code submission is decomposed into a set of node-specific, order-preserving subtrees generated via depth-first, preorder traversal. Each subtree corresponds to a semantically meaningful code fragment, such as loops, conditional blocks, or method invocations. These subtrees are transformed into vector representations through two complementary embedding operations: node embeddings, which encode individual nodes within a subtree, and subtree embeddings, which capture subtree-level structural information. The node embeddings within each subtree are first aggregated, and the resulting vector is concatenated with the corresponding subtree embedding to form a unified subtree representation. The resulting set of subtree vectors is then passed through an attention mechanism that assigns higher weights to code fragments most informative of a student’s understanding or programming concepts. The aggregated representation of weighted subtrees is originally fed into a neural network for classification, predicting whether a student is likely to succeed or struggle. In this work, however, we do not employ SANN in an end-to-end classification setting. Instead, we extract the intermediate code representations \(z_t\) produced by the fully connected layer following the attention mechanism and softmax normalization, and use these vectors along with correctness vector \(x_t\) as inputs to the LSTM model. It is to be noted here that unlike Code2Vec, which is a pretrained encoding model, SANN directly learns from the same client populations that participate in federated training for knowledge tracing. Therefore, to ensure that the resulting code representations reflect cohort-specific characteristics, we federate the SANN pipeline itself with the client data before the rest of the FL pipeline is combined.

3.3.3 FedsrcML-DKT

srcML-DKT extends this line of work by adopting srcML as the underlying code representation, enabling the modeling of uncompilable code submissions and thereby capturing syntactic errors that arise as students iteratively refine their solutions [27]. By replacing AST-based representations with srcML, as shown in Figure 2(c), this framework can parse code fragments that fail to compile and convert them into XML-based structures [7]. These representations can encode both syntactically correct and incorrect code by wrapping source text within XML tags that describe structural elements, while preserving erroneous or incomplete constructs textually. As a result, srcML-DKT incorporates all student submissions, rather than discarding non-compilable attempts. Following a pipeline analogous to Code2Vec, the srcML representations are subsequently transformed into fixed-length vector embeddings \(z_t\), which are then provided as inputs, along with correctness vector \(x_t\), to the LSTM-based KT model in FL setting. This design allows to model the evolution of student coding strategies across attempts while maintaining data privacy and retaining fine-grained syntactic information, even in the presence of compile errors.

3.4 Personalization of FL Models via pFL

Although both datasets originate from the same CS1 course and share identical feature definitions, the two semesters correspond to different student cohorts. Variations in prior experience, learning pace, and coding behavior lead to systematic shifts in interaction patterns and code structures, resulting in non-IID feature distributions across semesters, confirmed by statistically significant differences in a Kolmogorov–Smirnov (KS) test. Under such cohort-induced heterogeneity, a single global federated model may not perform uniformly across semesters, motivating the use of pFL to adapt to local data distributions. We adopted Federated Adaptive Local Aggregation (FedALA) [46], a pFL approach that extends vanilla FedAvg by introducing a personalized model synchronization mechanism at each client, as shown in Figure 3. This choice is motivated by FedALA’s superior capability of sufficiently adapting to local settings than simple post-hoc fine-tuning of the federated model [13].

While local training and global aggregation follow the standard FedAvg procedure, FedALA replaces direct overwriting of local models with an adaptive local aggregation (ALA) step. Let \(\theta ^{(\tau )}\) denote the global model parameters broadcast by the server at global aggregation round \(\tau \), and let \(\theta _k^{(\tau -1)}\) denote the final local model parameters of client \(k\) obtained at the end of the previous round. Rather than initializing the local model as \(\theta _k^{(\tau ),0} \leftarrow \theta ^{(\tau )}\), FedALA initializes the local parameters by adaptively combining the previous local model with the received global model:

where \(\lambda _k\) is a learnable, element-wise aggregation weight matrix constrained to the interval \([0,1]\), and \(\odot \) denotes element-wise multiplication. Instead of initializing the local model with the received global model, this adaptive aggregation allows each client to selectively incorporate globally shared information while suppressing updates that are less relevant to its local data distribution. FedALA further exploits the hierarchical structure of deep neural networks by applying adaptive aggregation only to the top \(l\) layers of the model, while the remaining lower layers (assumed to encode more general representations) are directly inherited from the global model without modification. FedALA learns \(\lambda _k\) using a fraction \(\zeta \) of each client’s local dataset. This design balances global knowledge sharing with client-specific adaptation. We tune the FedALA hyperparameters through grid search, with \(l \in \{1,2\}\) and \(\zeta \in [5,100]\) with increments of \(5\).

4. RESULTS & DISCUSSIONS

To evaluate the prediction task, we considered all student attempts across five programming assignments (A1-A5) and addressed our two research questions (RQ), described below. We measure predictive performance using area under curve (AUC): as programming datasets are highly imbalanced, AUC is more informative than accuracy, with \(0.5\) indicating random prediction and \(1\) indicating perfect discrimination. To answer RQ2, in addition to embedding-based representations, we report results for a non-embedding TF-IDF [34] method that represents code as weighted word-frequency vectors capturing surface-level patterns only.

RQ1: What does federated code representation performance difference mean in code-aware KT model performance? Figure 4 shows assignment-level AUC differences between Code-DKT and SANN-DKT and srcML-DKT and SANN-DKT under individual (i.e., sole local training) and federated settings, where positive values indicate higher AUC for SANN-DKT framework. As a reference, we used results reported by Hoq et al.[12] on Spring 2019 dataset, and examined whether SANN-DKT outperforming Code-DKT remains consistent under privacy-preserving FL. We replicated the same analysis on Fall 2019 dataset and the rank persisted. Further, we also implemented srcML-DKT and observed srcML-DKT performing better than SANN-DKT in most cases. Our results in Figure 4 top row indicate that FedSANN-DKT largely retains its code representation-driven performance advantage over FedCode-DKT, achieving significantly higher AUC values, confirmed by a paired Wilcoxon signed-rank test (\(p<0.05\)). Bottom row indicates similar outcomes between FedsrcML-DKT and FedSANN-DKT, further demonstrating that FL does not fundamentally disrupt the performance advantages conferred by richer code representations. However, this phenomenon is not uniform across all assignments. We observe small, assignment-specific reversals in relative performance, primarily on simpler tasks where overall AUC is already high and solution strategies exhibit limited structural diversity. This effect is most evident for A1 in top-left subfigure, where near-saturated performance constrains the marginal benefit of SANN’s neighborhood-based structural encoding over simpler path-based representations, making model rankings more sensitive to cohort-induced variation amplified through federated aggregation. In contrast, as assignments become conceptually more complex (A2–A5), and understanding code organization (i.e., combining multiple constructs and managing program control flow) is critical, representations that explicitly encode program structure more frequently outperform simpler path-based encodings under federated setting, resulting in \(AUC_{FedSANN-DKT} > AUC_{FedCode-DKT}\). This pattern is consistent with the performance hierarchy reported in individual training settings [12], unveiling that FL tends to preserve code representation rank with observed variations influenced by task complexity and data sparsity. These findings motivate a closer examination of how personalization interacts with representation choice under FL, which we investigate next in the context of RQ2.

RQ2: How does model personalization affect federated code representation across semesters/clients? Personalization via FedALA substantially improves cross-semester predictive accuracy in all code-aware federated KT models, with gains varying by assignment difficulty and code representation, as shown in Figure 5. Although TF-IDF is not a focus of our method, its inclusion here underscores that personalization by itself is insufficient; meaningful and consistent improvements in federated settings emerge primarily when personalization operates over embedding-based, code-aware representations. In contrast, embedding-based approaches show larger and more consistent personalization gains, with FedCode-DKT and FedSANN-DKT exhibiting improved alignment across semesters, particularly on later assignments. The effect is most pronounced for FedsrcML-DKT, where personalization leads to substantial and stable gains on structurally complex assignments (A4–A5) and reduces cross-semester performance gaps. A paired t-test shows that these sets of improvements achieved by FedALA over FedAvg are statistically significant (\(p < 0.05\)). These results suggest that though improvement magnitude varies based on task complexity, personalization does not degrade performance, instead it has the potential to better adapt to cohort-specific patterns in code-aware KT, while preserving privacy.

5. IMPLICATIONS FOR EDM & LIMITATIONS

Informed by our findings, personalized federated model generates actionable insights that instructors can use to identify students who may be struggling early in a privacy-preserving manner, particularly those showing repeated errors or inefficient coding patterns. These patterns reflect how students iteratively refine their problem-solving strategies over time through repeated attempts, errors, and code revisions.

However, two limitations should be noted despite promising results. First, our analysis is based on a single programming language Java. Since programming languages differ in syntax and structure, the observed performance trends may not generalize directly to other contexts such as Python or C++. Second, we model FL using two semesters of the same course from one institution. While there are some cohort differences, the overall learning environment remains similar. This limits the level of data heterogeneity compared to real-world federated settings that involve multiple institutions and more diverse student populations. Consequently, the extent to which the model performs consistently across diverse learner groups remains an open question. The close connection between fairness and model usefulness in human-centered applications [15] highlights the need for future evaluations on more diverse, cross-institutional datasets beyond predictive performance alone.

6. CONCLUSION & FUTURE WORK

In this paper, we studied how different code-aware knowledge tracing (KT) models behave under federated learning (FL), with a particular focus on code representation and model personalization. To this end, we present the first integration of personalized FL (pFL) with code embedding methods as an initial step toward privacy-preserving student code modeling and improved performance prediction. Using semester-level federated training on programming course data, we show that code representation type continues to influence predictive performance under FL, which pFL can further improve besides reducing cross-semester performance disparities through semester-specific adaptation. From a pedagogical perspective, this framework enables equitable and privacy-preserving instructional support regarding code-level misunderstanding. Our findings also indicate that retaining uncompilable code and incomplete attempts, which reflect authentic learning processes, leads to the strongest performance among the evaluated representations in pFL settings.

A key challenge in programming education datasets is the prevalence of incorrect submissions relative to correct ones, making it difficult for models to learn from the relatively few correct attempts. This limitation motivates future exploration of federated few-shot learning method to ensure that infrequent but informative demonstrations of understanding are not overlooked. Further, we plan to include features derived from student and system engagement data in our analysis, which our preliminary explorations suggest may capture insights of both coding and learning behavior. In practice, this will allow instructors to distinguish between students who struggle conceptually and those who are actively engaging but iterating through errors only, enabling more targeted instructional support. While our study focuses on a limited set of representations and course offerings, it highlights key design considerations for deploying privacy-preserving learning analytics in programming education, and motivates future research on representation-aware federated KT with programming-specific modeling choices, broader datasets, and richer FL personalization strategies.

7. ACKNOWLEDGMENT

This work was supported in part by the U.S. National Science Foundation (NSF) under Grant No. SaTC-2513164, ECCS-2512911, ECCS-2543754, and DRL-2418658.

References

1. Family educational rights and privacy act (FERPA). https://epic.org/family-educational-rights-and-privacy-act-ferpa/, Online; accessed 19 Feb. 2025.
2. U. Alon, M. Zilberstein, O. Levy, and E. Yahav. code2vec: Learning distributed representations of code. Proceedings of the ACM on Programming Languages, 3(POPL):1–29, 2019.
3. S. Bhattacharya, P. Vyas, S. Yarradoddi, B. Dasari, S. Sumathy, R. Kaluri, S. Koppu, D. J. Brown, M. Mahmud, and T. R. Gadekallu. Towards smart education in the industry 5.0 era: A mini review on the application of federated learning. In 2023 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), pages 0602–0608. IEEE, 2023.
4. H. Cen, K. Koedinger, and B. Junker. Learning factors analysis–a general method for cognitive model evaluation and improvement. In International conference on intelligent tutoring systems, pages 164–175. Springer, 2006.
5. Y.-W. Chu, S. Hosseinalipour, E. Tenorio, L. Cruz, K. Douglas, A. Lan, and C. Brinton. Mitigating biases in student performance prediction via attention-based personalized federated learning. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 3033–3042, 2022.
6. Y.-W. Chu, S. Hosseinalipour, E. Tenorio, L. Cruz, K. Douglas, A. S. Lan, and C. G. Brinton. Multi-layer personalized federated learning for mitigating biases in student predictive analytics. IEEE Transactions on Emerging Topics in Computing, X:1–15, 2024.
7. M. L. Collard, M. J. Decker, and J. I. Maletic. srcml: An infrastructure for the exploration, analysis, and manipulation of source code: A tool demonstration. In 2013 IEEE International conference on software maintenance, pages 516–519. IEEE, 2013.
8. A. T. Corbett and J. R. Anderson. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4(4):253–278, 1994.
9. Z. Duan, N. Fernandez, A. B. L. Narayanan, M. Hassany, R. S. de Alencar, P. Brusilovsky, B. Akram, and A. Lan. Automated knowledge component generation and knowledge tracing for coding problems. arXiv preprint arXiv:2502.18632, 2025.
10. S. H. Edwards and K. P. Murali. Codeworkout: short programming exercises with built-in data collection. In Proceedings of the 2017 ACM conference on innovation and technology in computer science education, pages 188–193, 2017.
11. M. Hoq, S. R. Chilla, M. Ahmadi Ranjbar, P. Brusilovsky, and B. Akram. Sann: Programming code representation using attention neural network with optimized subtree extraction. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 783–792, 2023.
12. M. Hoq, A. Rao, R. Jaishankar, K. Piryani, N. Janapati, J. Vandenberg, B. Mott, N. Norouzi, J. Lester, and B. Akram. Automated identification of logical errors in programs: Advancing scalable analysis of student misconceptions. In C. Mills, G. Alexandron, D. Taibi, G. L. Bosco, and L. Paquette, editors, Proceedings of the 18th International Conference on Educational Data Mining, pages 90–103, Palermo, Italy, July 2025. International Educational Data Mining Society.
13. A. P. Hridi, M. Hoq, Z. Gao, C. Lynch, R. Sahay, S. Hosseinalipour, and B. Akram. Privacy-preserving distributed link predictions among peers in online classrooms using federated learning. In C. Mills, G. Alexandron, D. Taibi, G. L. Bosco, and L. Paquette, editors, Proceedings of the 18th International Conference on Educational Data Mining, pages 276–288, Palermo, Italy, July 2025. International Educational Data Mining Society.
14. A. P. Hridi, R. Sahay, S. Hosseinalipour, and B. Akram. Revolutionizing ai-assisted education with federated learning: a pathway to distributed, privacy-preserving, and debiased learning ecosystems. In Proceedings of the AAAI Symposium Series, volume 3, pages 297–303, 2024.
15. A. P. Hridi and B. Watson. Are fair machine learning models more useful? In International Conference on Human-Computer Interaction, pages 38–53. Springer, 2024.
16. T. Huang, X. Ou, H. Yang, S. Hu, J. Geng, J. Hu, and Z. Xu. Remembering is not applying: Interpretable knowledge tracing for problem-solving processes. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 3151–3159, 2024.
17. P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et al. Advances and open problems in federated learning. Foundations and trends® in machine learning, 14(1–2):1–210, 2021.
18. M. A. Kia and A. Koliousis. Ackt: Self-attentive convolutional model for knowledge tracing. Procedia Comput. Sci., 246(C):2723–2732, Jan. 2024.
19. D. Kim, S. Kim, and Y. Jo. Knowledge tracing in programming education integrating students’ questions. In W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 27703–27718, Vienna, Austria, July 2025. Association for Computational Linguistics.
20. K. R. Koedinger, R. S. Baker, K. Cunningham, A. Skogsholm, B. Leber, and J. Stamper. A data repository for the edm community: The pslc datashop. Handbook of educational data mining, 43:43–56, 2010.
21. S. Lamsiyah, A. Nourbakhsh, S. El-Amrany, and C. Schommer. Privacy-preserving federated learning for student dropout prediction: Enhancing model transparency with explainable ai. In International Conference on Artificial Intelligence in Education, pages 324–332. Springer, 2025.
22. E. Latif and X. Zhai. Privacy-preserved automated scoring using federated learning for educational research. In International Conference on Artificial Intelligence in Education, pages 135–143. Springer, 2025.
23. Y. Liang, T. Peng, Y. Pu, and W. Wu. Help-dkt: an interpretable cognitive model of how students learn programming based on deep knowledge tracing. Scientific Reports, 12(1):4012, 2022.
24. H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In A. Singh and J. Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), volume 54 of Proceedings of Machine Learning Research, pages 1273–1282, Fort Lauderdale, FL, USA, Apr. 2017. PMLR.
25. H. Nakagawa, Y. Iwasawa, and Y. Matsuo. Graph-based knowledge tracing: Modeling student proficiency using graph neural network. In IEEE/WIC/ACM International Conference on Web Intelligence, WI ’19, page 156–163, New York, NY, USA, 2019. Association for Computing Machinery.
26. S. Pandey and G. Karypis. A self-attentive model for knowledge tracing. In C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, editors, Proceedings of the 12th International Conference on Educational Data Mining, pages 384–389. International Educational Data Mining Society, 2019.
27. M. Pankiewicz, Y. Shi, and R. S. Baker. srcml-dkt: Enhancing deep knowledge tracing with robust code representations from srcml. In C. Mills, G. Alexandron, D. Taibi, G. L. Bosco, and L. Paquette, editors, Proceedings of the 18th International Conference on Educational Data Mining, pages 541–548. International Educational Data Mining Society, July 2025.
28. P. I. Pavlik, H. Cen, and K. R. Koedinger. Performance factors analysis –a new alternative to knowledge tracing. In Proceedings of the 2009 Conference on Artificial Intelligence in Education: Building Learning Systems That Care: From Knowledge Representation to Affective Modelling, page 531–538, NLD, 2009. IOS Press.
29. C. Piech, J. Bassen, J. Huang, S. Ganguli, M. Sahami, L. J. Guibas, and J. Sohl-Dickstein. Deep knowledge tracing. Advances in neural information processing systems, 28, 2015.
30. A. Poh, A. P. Hridi, J. Barria-Pineda, P. Brusilovsky, and B. Akram. Example explorers and persistent finishers: Exploring student practice behaviors in a python practice system. In Proceedings of the 9th Educational Data Mining in Computer Science Education (CSEDM) Workshop, volume 4019 of CEUR Workshop Proceedings, Sicily, Italy, July 2025.
31. S. Pu, G. Converse, and Y. Huang. Deep performance factors analysis for knowledge tracing. In International Conference on Artificial Intelligence in Education, pages 331–341. Springer, 2021.
32. S. Pu, M. Yudelson, L. Ou, and Y. Huang. Deep knowledge tracing with transformers. In International conference on artificial intelligence in education, pages 252–256. Springer, 2020.
33. Y. Qin, M. Li, and J. Zhu. Privacy-preserving federated learning framework in multimedia courses recommendation. Wireless Networks, 29(4):1535–1544, 2023.
34. G. Salton and M. J. McGill. Introduction to Modern Information Retrieval. McGraw-Hill, New York, NY, 1988.
35. D. Sengupta, S. S. Khan, S. Das, and D. De. Fedel: Federated education learning for generating correlations between course outcomes and program outcomes for internet of education things. Internet of Things, page 101056, 2024.
36. M. Shaka, D. Carraro, and K. Brown. Error tracing in programming: a path to personalised feedback. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 330–342, 2024.
37. S. Shen, Q. Liu, Z. Huang, Y. Zheng, M. Yin, M. Wang, and E. Chen. A survey of knowledge tracing: Models, variants, and applications. IEEE Transactions on Learning Technologies, 17:1858–1879, 2024.
38. Y. Shi, M. Chi, T. Barnes, and T. Price. Code-DKT: A code-based knowledge tracing model for programming tasks. In A. Mitrovic and N. Bosch, editors, Proceedings of the 15th International Conference on Educational Data Mining, pages 50–61, Durham, United Kingdom, July 2022. International Educational Data Mining Society.
39. R. Sinha and V. Jaglan. A federated learning approach for personalized online course recommendation using hybrid recommender strategies. In 2025 International Conference on Computing and Communications (COMPUTINGCON), pages 1–7. IEEE, 2025.
40. S. Smith, A.-B. Lekshmi Narayanan, A. P. Hridi, R. Sampaio de Alencar, B. Akram, A. Hellas, J. Leinonen, P. Brusilovsky, and N. Norouzi. Knowledge component-driven alignment of cs1 textbooks and exercises. In Proceedings of the 57th ACM Technical Symposium on Computer Science Education V. 2, pages 1533–1534, 2026.
41. J. Wu, Z. Huang, Q. Liu, D. Lian, H. Wang, E. Chen, H. Ma, and S. Wang. Federated deep knowledge tracing. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM ’21, page 662–670, New York, NY, USA, 2021. Association for Computing Machinery.
42. C.-K. Yeung. Deep-irt: Make deep learning based knowledge tracing explainable using item response theory. In C. F. Lynch, A. Merceron, M. Desmarais, and R. Nkambou, editors, Proceedings of the 12th International Conference on Educational Data Mining, pages 683–686. International Educational Data Mining Society, 2019.
43. S. Yoneda, V. Svabensky, G. Li, D. Deguchi, and A. Shimada. Ranking-based at-risk student prediction using federated learning and differential features. In C. Mills, G. Alexandron, D. Taibi, G. L. Bosco, and L. Paquette, editors, Proceedings of the 18th International Conference on Educational Data Mining, pages 289–302, Palermo, Italy, July 2025. International Educational Data Mining Society.
44. M. V. Yudelson, K. R. Koedinger, and G. J. Gordon. Individualized bayesian knowledge tracing models. In International conference on artificial intelligence in education, pages 171–180. Springer, 2013.
45. D. Zhang, Q. Niu, T. Wang, Y. Hou, J. Wu, C. Zhang, and A. Stefanidis. Kgnn-kt: Enhancing knowledge tracing in programming education through llm-extracted knowledge graphs. In International Conference on Intelligent Computing, pages 137–147. Springer, 2025.
46. J. Zhang, Y. Hua, H. Wang, T. Song, Z. Xue, R. Ma, and H. Guan. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11237–1fedl2p1244, 2023.
47. J. Zhang, X. Shi, I. King, and D.-Y. Yeung. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th international conference on World Wide Web, pages 765–774, 2017.
48. S. Zhao, C. Wang, and S. Sahebi. Transition-aware multi-activity knowledge tracing. In 2022 IEEE International Conference on Big Data (Big Data), pages 1760–1769. IEEE, 2022.

© 2026 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.