Research Paper#Explainable Recommendation, LLMs, Factuality, Evaluation🔬 ResearchAnalyzed: Jan 3, 2026 15:36
Factual Consistency of Explainable Recommendation Models
Published:Dec 30, 2025 17:25
•1 min read
•ArXiv
Analysis
This paper addresses a crucial issue in explainable recommendation systems: the factual consistency of generated explanations. It highlights a significant gap between the fluency of explanations (achieved through LLMs) and their factual accuracy. The authors introduce a novel framework for evaluating factuality, including a prompting-based pipeline for creating ground truth and statement-level alignment metrics. The findings reveal that current models, despite achieving high semantic similarity, struggle with factual consistency, emphasizing the need for factuality-aware evaluation and development of more trustworthy systems.
Key Takeaways
- •Explainable recommendation models often generate explanations that are not factually consistent with the evidence.
- •A new framework is introduced to evaluate the factual consistency of these models.
- •Current models show a significant gap between fluency and factuality.
- •Factuality-aware evaluation is crucial for building trustworthy recommendation systems.
Reference
“While models achieve high semantic similarity scores (BERTScore F1: 0.81-0.90), all our factuality metrics reveal alarmingly low performance (LLM-based statement-level precision: 4.38%-32.88%).”