LLMs Fall Short for Learner Modeling in K-12 Education
Analysis
This paper highlights the limitations of using Large Language Models (LLMs) alone for adaptive tutoring in K-12 education, particularly concerning accuracy, reliability, and temporal coherence in assessing student knowledge. It emphasizes the need for hybrid approaches that incorporate established learner modeling techniques like Deep Knowledge Tracing (DKT) for responsible AI in education, especially given the high-risk classification of K-12 settings by the EU AI Act.
Key Takeaways
- •LLMs alone are not as effective as established learner modeling techniques (e.g., DKT) for assessing student knowledge in K-12 education.
- •LLMs struggle with temporal coherence and produce inconsistent mastery updates.
- •Responsible tutoring requires hybrid frameworks that combine LLMs with learner modeling.
- •Fine-tuning LLMs improves performance but still falls short of DKT and requires significant computational resources.
Reference / Citation
View Original"DKT achieves the highest discrimination performance (AUC = 0.83) and consistently outperforms the LLM across settings. LLMs exhibit substantial temporal weaknesses, including inconsistent and wrong-direction updates."