Large Language Models Approach Expert Pedagogical Quality in Math Tutoring but Differ in Instructional and Linguistic Profiles
Analysis
This research paper investigates the effectiveness of large language models (LLMs) in math tutoring by comparing their performance to expert and novice human tutors. The study focuses on both instructional strategies and linguistic characteristics, revealing that LLMs achieve comparable pedagogical quality to experts but employ different methods. Specifically, LLMs tend to underutilize restating and revoicing techniques, while generating longer, more lexically diverse, and polite responses. The findings highlight the potential of LLMs in education while also emphasizing the need for further refinement to align their strategies more closely with proven human tutoring practices. The correlation analysis between specific linguistic features and perceived quality provides valuable insights for improving LLM-based tutoring systems.
Key Takeaways
- •LLMs can achieve expert-level pedagogical quality in math tutoring.
- •LLMs differ from human experts in instructional and linguistic strategies.
- •Restating and revoicing are key strategies underutilized by LLMs.
“We find that large language models approach expert levels of perceived pedagogical quality on average but exhibit systematic differences in their instructional and linguistic profiles.”