Interpretable Knowledge Tracing: Simple and Efficient Student Modeling with Causal Relations
Sein MinnJill-Jênn VieKoh TakeuchiHisashi KashimaFeida Zhu
Proposes an interpretable knowledge tracing model that combines skill mastery, cross-skill learning transfer, and problem difficulty within a Tree-Augmented Naive Bayes classifier to outperform deep learning approaches while providing clear causal explanations of student learning.
Online and intelligent tutoring platforms rely heavily on knowledge tracing—the ability to model a learner's skill mastery over time and predict whether they will answer future exercises correctly. While modern deep-learning models achieve strong predictive accuracy, they function as complex "black boxes" with tens of thousands of opaque parameters. This lack of interpretability prevents educators and system designers from understanding why a student succeeded or failed, hindering the delivery of meaningful, diagnostic instructional interventions.
The article demonstrates a novel student modeling framework called Interpretable Knowledge Tracing (IKT). The main objective is to evaluate whether a transparent, probabilistic graphical model can outperform complex neural networks in predicting student performance while providing clear, psychologically grounded causal explanations.
To achieve this, the approach uses standard data mining methods to extract three interpretable latent factors from student interaction logs: individual skill mastery (estimated via traditional sequential modeling), dynamic ability profiles that capture cross-skill learning transfer (grouped via clustering algorithms over rolling 20-attempt windows), and problem difficulty (scaled from 1 to 10 based on historical first-attempt failure rates). These features are integrated into a Tree-Augmented Naive Bayes classifier, which captures the dependencies among features to forecast future answers. The approach was tested against established baseline models across three large public tutoring datasets comprising up to 28,834 students and over 2.5 million interaction records.
The evaluation produced several key findings. First, IKT consistently outperformed or matched all state-of-the-art deep-learning models across the benchmark datasets, reaching an average Area Under the Curve (AUC) of 0.805 and lower error rates. Second, ablation analysis revealed that problem difficulty is by far the most influential factor in forecasting success, driving an absolute predictive improvement of 11.2% to 15.7% in AUC. Third, tracking dynamic student ability profiles provided modest additional predictive gains (up to 1.4% in AUC) by accounting for general learning transfer across different skills.
These results demonstrate that organizations deploying intelligent tutoring systems do not need to sacrifice model interpretability to achieve cutting-edge predictive power. By replacing resource-heavy deep neural networks with probabilistic graphical models, educational platforms can significantly reduce computational training costs, lower infrastructure overhead, and improve operational transparency. Furthermore, the model's conditional probability structure provides clear causal pathways, allowing systems to diagnose whether a student's incorrect answer stemmed from a genuine skill deficiency or an exceptionally difficult problem.
For educational technology leaders and system developers, the article supports adopting feature-driven probabilistic models like IKT within personalized curriculum engines. Future work should focus on field-testing these causal insights in live instructional environments to guide real-time adaptive interventions. Platform managers should, however, note the primary operational limitation: the model requires discrete data bins, relies on having at least a few prior student attempts to reliably estimate item difficulty, and initiates ability profiling only after an initial baseline of student activity.
- Paper: The Mythos of Model Interpretability, Zachary C. Lipton (2016). This paper establishes the foundational conceptual framework distinguishing transparent, interpretable-by-design models from post-hoc explanations, motivating the design philosophy behind Interpretable Knowledge Tracing.
- Paper: Concept Bottleneck Models, Pang Wei Koh et al. (2020). This work introduces the methodology of using interpretable intermediate latent concepts to drive final predictions, directly informing how student mastery and difficulty profiles are extracted for transparent modeling.
- Paper: A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science, Clayton Cohn et al. (2024). This study advances interpretable student modeling by applying transparent reasoning frameworks to automated formative assessment and qualitative feedback in real-world educational domains.
