Abstract

Knowledge Tracing (KT) models are widely used in adaptive learning systems to estimate students’ evolving knowledge states. Despite recent advances in deep KT models, many approaches exhibit poor probabilistic calibration, producing overconfident predictions that misrepresent students’ knowledge. This paper presents Counterfactual Item-Invariance Regularized Knowledge Tracing (CIR-KT), a bias-aware approach for calibrated knowledge tracing. For each interaction the method constructs an item-level counterfactual, substituting the presented item while holding the observed response fixed, and enforces consistency between the factual and counterfactual latent states through a regularization objective. By reducing the model’s reliance on item-identity shortcuts, the framework improves probability calibration without altering the underlying KT architecture. On a severely miscalibrated private College of Engineering (COE) subset, counterfactual regularization substantially reduces Expected Calibration Error (ECE) and Brier score and improves ranking (AUC and PR-AUC) relative to the DeepIRT backbone, while on already well-calibrated public benchmarks its effect on discrimination and calibration is small. Skill-level analyses show that the calibration gains are broad-based across knowledge components rather than driven by a few items. These results indicate that item-level counterfactual regularization is a practical mechanism for improving calibration in knowledge tracing, supporting more dependable adaptive learning systems. At the validation-selected operating point, CIR-KT is the best-calibrated model among the knowledge-tracing backbones evaluated while remaining competitive in AUC, and it provides interpretable ability and difficulty estimates that attention baselines do not.