Encyclo-K: A New Benchmark for Evaluating LLMs

This paper introduces Encyclo-K, a novel benchmark for evaluating Large Language Models (LLMs). It addresses limitations of existing benchmarks by using knowledge statements as the core unit, dynamically composing questions from them. This approach aims to improve robustness against data contamination, assess multi-knowledge understanding, and reduce annotation costs. The results show that even advanced LLMs struggle with the benchmark, highlighting its effectiveness in challenging and differentiating model performance.