Novel Metric Reveals LLM Alignment Insights for Value-Oriented Evaluation

This research introduces an innovative approach to evaluating the alignment of Large Language Models (LLMs) with human values, leveraging survey responses. By introducing the 'self-correlation distance' metric, the study offers a powerful method to assess the consistency of LLM responses, paving the way for more robust and reliable evaluation frameworks. This advancement promises to refine how we understand and assess the ethical implications of Generative AI.

• •The study explores limitations in current methods of evaluating Large Language Models using social surveys.
• •A new metric, 'self-correlation distance', is introduced to assess consistency in LLM responses.
• •The research suggests best practices for future evaluations, including Chain of Thought prompting and sampling-based decoding.