LLM Ensemble Method for Response Selection
Analysis
This paper introduces LLM-PeerReview, an unsupervised ensemble method for selecting the best response from multiple Large Language Models (LLMs). It leverages a peer-review-inspired framework, using LLMs as judges to score and reason about candidate responses. The method's key strength lies in its unsupervised nature, interpretability, and strong empirical results, outperforming existing models on several datasets.
Key Takeaways
Reference
“LLM-PeerReview is conceptually simple and empirically powerful. The two variants of the proposed approach obtain strong results across four datasets, including outperforming the recent advanced model Smoothie-Global by 6.9% and 7.3% points, respectively.”