∗Corresponding author

ABSTRACT

Educational prediction models are increasingly used to identify students who may need early academic support. When such models are trained with privacy-preserving methods, evaluation often focuses on average predictive performance and measured privacy leakage. However, this can overlook an important deployment concern: private training may also change how prediction errors are distributed across student groups. We examine this issue using two public educational datasets. OULAD serves as the primary dataset because it supports both privacy auditing and group-level error analysis, while UCI697 is used only as a secondary consistency check for the relationship between private training, predictive quality, and measured privacy leakage. Across both datasets, we compare standard non-private training with Differentially Private Stochastic Gradient Descent (DP-SGD). We evaluate predictive quality using test AUC, measured privacy leakage using a loss-based membership inference attack, and, on OULAD, group-level error patterns using worst-group gaps in true positive rate and false positive rate. Across both datasets, stronger private training reduces predictive quality more clearly than it changes the measured privacy-audit outcome. On OULAD, under the fixed threshold used for controlled comparison, privacy-preserving training also increases group-level error gaps. These results suggest that evaluating student prediction models only through average predictive performance and measured privacy leakage can miss an important part of the effect of privacy-preserving training. Educational prediction models trained with privacy-preserving methods should be evaluated jointly in terms of predictive quality, measured privacy leakage, and group-level error patterns, especially when they are used to inform student support decisions. Our code is available on GitHub: https://github.com/GEMLabHKU/DP-FAIRNESS/tree/main

Keywords

1. INTRODUCTION

Prediction models are increasingly used to identify students who may be at risk of failure, withdrawal, or dropout [1, 2, 3]. These models are sometimes trained on sensitive educational records, including demographic information, academic histories, and learning activity traces. The collection and use of such records raise concerns about student privacy, consent, transparency, and institutional responsibility [4]. Privacy protection is therefore needed because trained machine learning models may reveal information about the records used during training, including whether a particular student’s record was included [5, 6].

Prior work has developed privacy-preserving training methods such as DP-SGD [7, 8]. Evaluations of such methods commonly consider predictive performance and measured privacy leakage. Predictive performance describes how well a model distinguishes between students with different outcomes, whereas measured privacy leakage examines whether information about individual training records remains detectable from the trained model. For example, a membership inference attack tests whether an attacker can determine whether a particular user record was included in the training data [5, 6]. One common audit is membership inference, which tests whether an attacker can determine whether a particular student record was included in the training data. Membership inference does not necessarily reveal a student’s identity or complete educational record. However, when an attacker already has auxiliary information about a student, evidence of training-set membership may strengthen an association between that student and sensitive information, such as course participation, academic difficulty, dropout risk, or receipt of educational support. For this reason, reducing measured privacy leakage is an important goal of privacy-preserving training. However, privacy and utility alone may still provide an incomplete view of model behaviour in deployment [9]. A model may maintain usable predictive performance and show little detectable membership signal under a selected privacy attack, while still changing how prediction errors are distributed across student groups.

In practical implementation, prediction models may influence which students are identified for support, which students are overlooked, and which students are incorrectly flagged for intervention. Therefore, even when a privacy-preserving model retains usable overall predictive performance and shows little detectable membership signal under a selected privacy attack, it may still distribute prediction errors unevenly across student groups. This matters because uneven error patterns can create unequal consequences for students: some groups may be more likely to receive timely support, while others may be more likely to be missed or unnecessarily targeted for intervention. Thus, in addition to evaluating overall model quality and measured privacy leakage, privacy-preserving training should also be evaluated in terms of how prediction errors are distributed across student groups.

This study investigates this deployment concern by examining whether the introduction of privacy-preserving training changes not only predictive quality and measured privacy leakage, but also the distribution of prediction errors across student groups. Specifically, we compare standard non-private training with Differentially Private Stochastic Gradient Descent (DP-SGD), a widely used privacy-preserving training method that clips per-example gradients and adds noise during optimisation [7, 8]. We use OULAD as the primary dataset because it supports both privacy auditing and group-level error analysis, and we use UCI697 as a secondary case to test whether the relationship between privacy protection and predictive quality also appears in another educational setting [3, 10].

To evaluate individual-level privacy risk, we use a membership inference attack (MIA), which tests whether an attacker can distinguish a student record used for model training from a record that was not used for training [5, 6]. This attack provides one operational measure of whether information about an individual training record remains detectable from the trained model. To evaluate group-level consequences, we examine whether privacy-preserving training changes how prediction errors are distributed across student groups [1, 2]. These two concerns are commonly studied through different evaluation traditions. Privacy research often focuses on formal privacy guarantees or attack-based audits of individual records [5, 6], whereas educational prediction research commonly evaluates average predictive performance and examines group fairness as a separate dimension [11, 12]. Evaluating these dimensions separately can obscure an important deployment concern: a model may retain usable average predictive performance and show little detectable membership signal under a selected privacy attack, while still producing more uneven error patterns across student groups after privacy-preserving training is introduced. This study contributes an integrated evaluation perspective for privacy-preserving educational prediction by examining predictive quality, measured individual-level privacy leakage, and group-level error patterns within the same empirical comparison. This joint evaluation makes visible whether private training changes not only average model performance and detectable membership signals, but also which student groups are more likely to be correctly identified, overlooked, or incorrectly flagged for support. As introduced in Figure 1, our study is guided by two Research Questions (RQs):

• How does private training change predictive quality and measured privacy leakage in educational prediction?
• Does private training also change group-level error patterns in ways that would be missed by evaluation based only on utility and privacy audit?

2. METHOD

Our evaluation has three parts, each aligned with the paper’s central question: predictive quality, measured privacy leakage, and group-level error patterns. Predictive quality is evaluated using test AUC. Measured privacy leakage is evaluated using a loss-based membership inference attack. Group-level error patterns are evaluated on OULAD using worst-group gaps in true positive rate (TPR) and false positive rate (FPR).

2.1 Datasets and prediction tasks

We use two public educational datasets with different roles in the study. OULAD is the primary dataset because it is the only dataset in our experiments that supports all three parts of the analysis within a single setting: predictive quality, privacy auditing, and group-level error evaluation [3]. UCI697 is used only as a secondary consistency check for the relationship between private training, predictive quality, and measured privacy leakage [10].

The Open University Learning Analytics Dataset (OULAD) contains 32,593 student-course records with course outcomes and demographic attributes such as gender, region, age band, and education level [3]. The prediction task in this study is course failure or withdrawal. We use structured student and course features, including previous attempts, studied credits, and categorical variables converted into numeric form.

The UCI Student Dropout and Academic Success dataset (UCI697) contains 4,424 higher education records and 36 features related to enrolment background and academic progress [10]. The prediction target is dropout versus graduation. We do not use UCI697 for group-level error analysis in this study. For both datasets, all training conditions use the same preprocessing pipeline within each dataset, so comparisons are between training conditions rather than between different feature sets or preprocessing choices.

2.2 Training setup and split protocol

We compare standard non-private training with Differentially Private Stochastic Gradient Descent (DP-SGD) [7]. DP-SGD modifies ordinary gradient-based training in two steps: it clips each example’s gradient to bound the influence of any single training record, and then adds Gaussian noise to the aggregated clipped gradients during optimisation. In our experiments, the clipping norm is fixed at \(C=1.0\). We evaluate privacy budgets \(\epsilon \in \{1,5,10\}\) and set \(\delta = 1/n_{\text {train}}\), with smaller \(\epsilon \) corresponding to stronger privacy protection.

We use logistic regression as a simple linear baseline and a small multilayer perceptron (MLP-small) as the main nonlinear baseline for matched comparisons. MLP-small is chosen to compare private and non-private training under the same feature representation. Across conditions, the model family, feature inputs, data splits, and evaluation protocol are held fixed. The main change is whether DP-SGD is applied during training. All conditions are run with matched random seeds, and each reported result summarises five runs per condition. For each run, models are trained on a training split and evaluated on the same held-out test split for predictive quality, privacy, and, where applicable, group-level error analysis. This keeps the comparison focused on the effect of private training rather than on differences in data partitioning.

2.3 Privacy audit and group-level error evaluation

To evaluate measured privacy leakage, we use the loss-based membership inference attack proposed by Yeom et al. [6]. In this attack, the attacker uses prediction loss as the signal, with lower-loss examples treated as more likely to be training members. For each run, the attack compares training members with held-out non-members from the same split used for model evaluation. We report membership inference attack AUC, which measures how well the attack separates training members from non-members. An AUC close to \(0.5\) indicates near-random separation, whereas a larger value indicates stronger measured membership signal. We use this attack as one measured privacy audit under a specific threat model, rather than as a complete characterisation of privacy risk in educational prediction.

To evaluate group-level error patterns on OULAD, we compare two worst-group error gaps across gender groups. We use gender as a first group-based check because this attribute is complete and consistently represented in the OULAD records used here. The first metric is the worst-group gap in true positive rate, and the second is the worst-group gap in false positive rate. For a group \(g\), \(\text {TPR}_g\) is the proportion of positive cases in that group that the model correctly predicts as positive, and \(\text {FPR}_g\) is the proportion of negative cases in that group that the model incorrectly predicts as positive. We report

and

A larger TPR gap means that some groups are more likely than others to receive correct positive predictions, whereas a larger FPR gap means that some groups are more likely than others to receive incorrect positive predictions. All group-level error results use a fixed decision threshold of \(0.5\) to support controlled comparison across training conditions. This choice is intended to isolate changes in model behaviour rather than to represent an optimal deployment threshold.

3. RESULTS

3.1 RQ1: Privacy-preserving training on predictive quality and measured leakage

Table 1 reports the main OULAD results across five matched random seeds per condition. For RQ1, we examine two outcomes commonly used to evaluate private training: predictive quality and measured privacy leakage. Table 2 reports the corresponding utility–privacy patterns on UCI697 as a secondary consistency check.

We begin with predictive quality. On OULAD, the non-private MLP-small baseline achieves a test AUC of \(0.661 \pm 0.004\). Under DP-SGD with \(\epsilon =1\), test AUC decreases to \(0.644 \pm 0.002\), and partially recovers to \(0.649 \pm 0.002\) at \(\epsilon =5\) and \(\epsilon =10\). This pattern is consistent with the expected privacy–utility trade-off: stronger privacy protection produces the largest reduction in predictive quality, whereas looser privacy budgets recover some utility. UCI697 shows the same pattern. There, the non-private baseline achieves the highest test AUC, \(0.952 \pm 0.008\), DP-SGD at \(\epsilon =1\) yields the largest drop to \(0.919 \pm 0.012\), and performance improves again at \(\epsilon =5\) and \(\epsilon =10\). Across both datasets, private training therefore changes predictive quality in the expected direction.

The measured privacy-leakage results require a more cautious reading. On OULAD, the loss-based membership inference attack AUC remains close to random guessing in all conditions, including the non-private baseline. This is therefore better interpreted as evidence that the evaluated attack is already weak in this setting, rather than evidence that DP-SGD removes a strong privacy leakage signal. Additional analysis supports this interpretation: member and non-member scores are already highly similar before private training is applied. The UCI697 results support the same reading, with membership inference attack AUC remaining close to \(0.5\) across all conditions.

Taken together, the RQ1 results show that private training changes predictive quality more clearly than it changes the measured privacy-audit outcome. This means that evaluation based only on utility and privacy audit provides only a partial view of its effects.

3.2 RQ2: Privacy-preserving training on group-level error patterns

RQ2 examines the part of model behaviour that would be missed if evaluation were limited to predictive quality and privacy audit. This analysis is conducted only on OULAD, the only dataset in this study that supports group-level error evaluation. The answer is yes: under the fixed threshold used here, private training changes how prediction errors are distributed across groups.

Relative to the non-private MLP-small baseline, both worst-group error gaps increase under private training. At \(\epsilon =1\), the TPR gap rises from \(0.016 \pm 0.009\) to \(0.045 \pm 0.014\), and the FPR gap rises from \(0.009 \pm 0.002\) to \(0.064 \pm 0.014\). At \(\epsilon =5\) and \(\epsilon =10\), both gaps remain above the non-private baseline, although they are smaller than at \(\epsilon =1\). This indicates that the shift is not confined to the strictest privacy condition, even if it is most pronounced there.

This finding matters because the change in group-level error patterns appears alongside only a modest change in overall AUC. On OULAD, private training does not remove the model’s predictive usefulness, but it does alter how support-related errors are distributed across groups. A deployment decision based only on average predictive quality and privacy audit could therefore conclude that the private model remains broadly acceptable, while missing that the pattern of errors across groups has shifted.

The two gap types also have distinct practical interpretations. A larger TPR gap means that some groups are more likely than others to receive correct positive predictions, whereas a larger FPR gap means that some groups are more likely than others to receive incorrect positive predictions. In educational settings, these differences can translate into unequal access to intervention or unequal exposure to unnecessary intervention, even when the model still appears acceptable on average performance metrics.

Taken together with RQ1, these results show why utility and privacy audit alone are not sufficient for evaluating private training in educational prediction. In settings such as OULAD, private training changes not only overall utility, but also how support-related errors are distributed across groups.

4. DISCUSSION

This study shows that private training can change more than average predictive performance and measured privacy leakage. On OULAD, DP-SGD reduced predictive quality to some extent, while the evaluated loss-based membership inference attack remained close to random across both private and non-private conditions. When evaluation stopped at these two dimensions, the training conditions might appear similar under the selected privacy audit, even though the audit provides limited evidence because attack performance was already close to random for the non-private model. However, the same models produced larger worst-group TPR and FPR gaps under the fixed decision threshold used for comparison. This indicates that private training can alter the distribution of support-related prediction errors even when average predictive performance remains usable and the selected privacy audit detects little membership signal.

4.1 Implications

The practical importance of this finding lies in how educational prediction models are used. In student-risk modelling, predictions are often connected to advising, tutoring, outreach, or other forms of early support. A true positive can correspond to a student being correctly identified for support, whereas a false negative can correspond to a student being overlooked. Similarly, a false positive can expose students to unnecessary intervention or concern. Group-level shifts in TPR and FPR therefore matter because they may change which groups are more likely to receive support, be missed, or be incorrectly flagged. The contribution of this paper is to make this deployment issue visible in the evaluation of privacy-preserving educational prediction models.

This finding also has a methodological implication for educational data mining. Privacy, predictive performance, and group-level error behaviour are often evaluated as separate concerns. The present results suggest that this separation can be incomplete when prediction models are used to inform student support. In this setting, average utility and the evaluated privacy audit alone would not have shown that private training also changed who was more likely to be correctly identified, overlooked, or unnecessarily flagged. Evaluating these dimensions jointly therefore provides a more deployment-relevant view of privacy-preserving educational prediction.

4.2 Limitations and Future Work

Several limitations define the scope of the findings. First, the privacy audit uses only one loss-based membership inference attack, which detected little separation between members and non-members even before private training. The results therefore should not be interpreted as evidence that DP-SGD removed a strong membership signal or as a complete assessment of privacy risk. Future work should examine stronger and complementary attacks, including confidence-based, entropy-based, shadow-model, label-only, and white-box attacks. Second, the group-level analysis is limited to gender groups in OULAD and to worst-group TPR and FPR gaps. Future studies should examine additional attributes, intersectional groups, and complementary metrics such as FNR, positive prediction rate, positive predictive value, and calibration error. Third, the fixed threshold of \(0.5\) supports controlled comparison but does not represent all institutional intervention rules. Future work should test whether similar shifts appear across thresholds and realistic support-allocation policies. Fourth, the analysis documents group-level error shifts but does not identify their causes. Further work should examine whether these shifts arise from gradient clipping, added noise, subgroup size, class imbalance, or feature-distribution differences. Finally, the study uses two public datasets and relatively simple models. The findings should therefore be tested with larger datasets, richer model families, and deployment settings where student identifiability, privacy risk, and intervention consequences can be evaluated more directly.

5. CONCLUSION

This paper investigated how privacy-preserving training changes predictive quality, measured privacy leakage, and group-level error patterns in educational prediction. Across two public datasets, stronger privacy-preserving training consistently reduced predictive quality, while the evaluated loss-based membership inference attack remained close to random. On OULAD, privacy-preserving training also changed worst-group TPR and FPR gaps across gender groups.

These findings show that evaluating privacy-preserving training only as a privacy–utility trade-off can provide an incomplete account of its deployment effects. A model may retain usable average predictive performance and show little measured leakage under a particular membership inference attack, while still changing how prediction errors are distributed across student groups. In educational settings, this matters because such errors may affect which students are correctly identified for support, which students are overlooked, and which students are unnecessarily flagged for intervention.

The main contribution of this paper is therefore to broaden the evaluation frame for privacy-preserving educational prediction. Rather than treating predictive quality, measured privacy leakage, and group-level error patterns as separate concerns, our results suggest that they should be examined jointly when student prediction models are used to inform support decisions. This joint evaluation is especially important in contexts where privacy-preserving training is adopted to protect student data, but the resulting models may still have unequal consequences across student groups.

6. ACKNOWLEDGMENTS

This work was supported by the Faculty Research Fund and by the grant from the URC (Grant No. 2401102970) at The University of Hong Kong.

7. REFERENCES

1. Qian Hu and Huzefa Rangwala. Towards fair educational data mining: A case study on detecting at-risk students. International Educational Data Mining Society, 2020.
2. Josh Gardner, Christopher Brooks, and Ryan Baker. Evaluating the fairness of predictive student models through slicing analysis. In Proceedings of the 9th international conference on learning analytics & knowledge, pages 225–234, 2019.
3. Jakub Kuzilek, Martin Hlosta, and Zdenek Zdrahal. Open university learning analytics dataset. Scientific Data, 4(1):170171, 2017.
4. Abelardo Pardo and George Siemens. Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3):438–450, 2014.
5. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017.
6. Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268–282. IEEE, 2018.
7. Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308–318, 2016.
8. Anika Kabir, Chandan Tankala, and Daniel Lowd. On the practicality of differential privacy for knowledge tracing. In Proceedings of the 18th International Conference on Educational Data Mining, pages 619–624, 2025.
9. Mélina Verger, Sébastien Lallé, François Bouchet, and Vanda Luengo. Is your model “MADD”’? a novel metric to evaluate algorithmic fairness for predictive student models. In Proceedings of the 16th International Conference on Educational Data Mining, pages 91–102, 2023.
10. Valentim Realinho, Mónica Vieira Martins, Jorge Machado, and Luís Baptista. Predict students’ dropout and academic success. UCI Machine Learning Repository, 2021.
11. René F. Kizilcec and Hansol Lee. Algorithmic fairness in education. In Wayne Holmes and Kaśka Porayska-Pomsta, editors, The Ethics of Artificial Intelligence in Education, pages 174–202. Routledge, 2022.
12. Hansol Lee and René F. Kizilcec. Evaluation of fairness trade-offs in predicting student success, 2020.

© 2026 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.