Factual Consistency of Explainable Recommendation Models

This paper addresses a crucial issue in explainable recommendation systems: the factual consistency of generated explanations. It highlights a significant gap between the fluency of explanations (achieved through LLMs) and their factual accuracy. The authors introduce a novel framework for evaluating factuality, including a prompting-based pipeline for creating ground truth and statement-level alignment metrics. The findings reveal that current models, despite achieving high semantic similarity, struggle with factual consistency, emphasizing the need for factuality-aware evaluation and development of more trustworthy systems.